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Abstract 


The  movement  of  information,  agents,  and  resources  is  a  crucial  part  of  co¬ 
operative  multiagent  systems:  decision  makers  must  receive  data  in  a  timely 
manner  to  make  good  decisions,  while  agents  and  resources  must  be  provided 
at  appropriate  locations  for  tasks  to  be  completed.  Flow  allocation  meets  these 
conditions  by  computing  paths  through  the  environment,  be  it  the  communica¬ 
tion  network  (for  data  or  software  agents)  or  the  physical  world  (for  embodied 
agents  or  physical  resources).  This  thesis  addresses  the  problem  of  allocating 
flows  when  the  environment  is  mutable,  either  by  the  agents  or  by  a  malicious 
adversary. 

In  this  thesis  I  represent  the  environment  as  a  graph  with  agents  and  tasks 
represented  by  source  and  sink  nodes,  respectively.  The  agents  are  partitioned 
into  groups,  and  the  flows  from  members  of  a  group  must  be  sent  to  the  same 
sink,  reflecting  cases  where  a  task  requires  multiple  agents  to  work  together 
or  a  decision  requires  multiple  inputs.  Capacity  and  cost  constraints  on  the 
graph  reflect  environmental  factors  affecting  the  feasibility  and  quality  of  al¬ 
locations.  I  prove  that  even  relatively  simple  problems  of  allocating  flows  for 
groups  in  graphs  are  not  just  hard  to  solve  optimally,  but  also  hard  to  approx¬ 
imate. 

In  some  cases  the  agent  team  can  add  additional  nodes  and  edges  to  the 
graph,  for  instance  by  deploying  unmanned  aerial  vehicles  (UAVs)  as  commu¬ 
nication  relays  to  supplement  a  ground-based  wireless  ad  hoc  network.  I  prove 
that  augmenting  the  network  optimally  is  intractable,  and  provide  algorithms 
for  solving  it  heuristically. 

In  other  cases  an  adversary  can  impair  the  agents  by  choosing  attacks  that 
increase  costs  to  the  agents.  Agents  must  then  choose  their  paths  through 
the  network  strategically,  reasoning  not  just  about  the  environmental  costs 
but  also  about  the  behavior  of  the  adversary.  I  formalize  this  interaction  as 
a  game  between  the  agent  team  and  the  adversary,  and  provide  polynomial 
time  algorithms  for  computing  equilibrium  strategies  in  simultaneous  zero- 
sum  games,  simultaneous  non-zero-sum  games  where  each  player  unilaterally 
incurs  costs  to  play  their  strategies,  and  sequential  non-zero-sum  games  where 
the  sender  can  commit  to  a  strategy. 
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Chapter  1 
Introduction 


A  key  feature  of  cooperative  multiagent  systems  is  the  movement  of  agents,  information, 
and  resources  in  order  to  facilitate  task  execution.  Performing  each  task  requires  a  combi¬ 
nation  of  agents,  information,  and  resources,  and  these  elements  must  generally  be  brought 
to  a  single  location  for  the  task  to  be  performed  successfully.  For  example,  in  a  disaster  re¬ 
sponse  application,  extinguishing  a  fire  (the  task)  requires  bringing  fire  trucks  (the  agents) 
and  water  (the  resource)  to  the  location  of  the  fire.  In  a  sensor  network  application,  sensor 
readings  (information)  from  multiple  sensors  (the  agents)  must  be  sent  to  a  base  station 
(the  resource)  for  data  fusion  to  track  a  target  (the  task).  Different  paths  can  impose  dif¬ 
ferent  costs  to  task  execution,  for  example  by  draining  more  battery  power  in  the  sensor 
network  or  delaying  arrival  of  the  fire  trucks  to  the  fire.  The  problem  of  choosing  paths 
for  the  movement  of  agents,  information,  and  resources  is  thus  crucial  because  it  governs 
both  what  tasks  can  be  performed  as  well  as  the  quality  with  which  they  are  performed. 

Network  flows  provide  a  natural  way  to  represent  this  problem.  The  environment  is  repre¬ 
sented  as  a  finite,  directed  graph  with  nodes  representing  locations  and  edges  representing 
allowable  transitions  between  locations.  This  graph  may  model  the  external,  physical  en¬ 
vironment,  such  as  the  buildings  and  road  network  of  a  city,  or  the  rooms  and  corridors  of  a 
building,  or  it  may  model  the  communication  network  between  agents  or  some  other  inter¬ 
nal  structure  of  the  team.  Source  nodes  represent  the  initial  locations  of  things  that  must 
move  through  the  graph,  such  as  agents  or  information,  while  sink  nodes  represent  the 
locations  of  task  execution.  Capacity  and  cost  constraints  on  edges  reflect  environmental 
factors  that  affect  feasibility  and  quality  of  allocations,  with  capacitated  graphs  best  suited 
for  modeling  communication  networks  and  costly  graphs  appropriate  for  both  physical 
and  communication  environments.  For  example,  in  a  physical  environment  costs  may 
model  distances  traveled  by  robots,  while  in  a  communication  environment  costs  reflect 
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the  battery  usage  for  transmitting  needed  data  over  communication  links,  while  capacity 
constraints  represent  limited  available  bandwidth  on  those  links.  The  problem  of  meeting 
the  requirements  for  task  execution  then  becomes  the  problem  of  allocating  flows  from  the 
source  nodes  to  the  sink  nodes. 

This  thesis  addresses  the  problem  of  flow  allocation  in  multiagent  teams  when  the  environ¬ 
ment  is  mutable,  either  by  the  agents  or  by  a  malicious  adversary.  The  type  of  modification 
depends  on  whether  the  environment  is  capacitated  or  costly.  In  capacitated  environments 
agents  may  relieve  bottlenecks  or  provide  connectivity  to  disconnected  parts  of  the  envi¬ 
ronment  by  augmenting  the  environment  with  additional  nodes  and  edges.  For  example, 
in  a  communication  network  this  may  be  achieved  by  deploying  unmanned  aerial  vehicles 
to  act  as  communication  relays  to  supplement  a  ground-based  ad  hoc  wireless  network. 
In  costly  environments,  an  adversary  can  impose  additional  costs,  for  example  by  deploy¬ 
ing  cameras  to  observe  the  movements  of  agents,  placing  obstacles  to  delay  agents,  or 
jamming  a  communication  network  to  force  retransmissions. 


1.1  Flow  Allocation  for  Multiagent  Teams 

A  common  approach  to  designing  multiagent  teams  is  to  separate  domain-level  task  knowl¬ 
edge  from  domain-independent  coordination  ability.  Domain-level  knowledge  is  typically 
represented  using  hierarchical  task  networks  (HTNs)  [18]  that  explicitly  specify  depen¬ 
dencies  betweens  tasks,  including  the  requirements  for  execution.  Tasks  in  an  HTN  may 
be  primitive  tasks  that  can  be  directly  executed,  goal  tasks  that  describe  desired  states  of 
the  world,  or  compound  tasks  that  are  made  up  of  other  tasks.  In  multiagent  systems, 
multiple  agents  must  often  work  together  to  achieve  goal  tasks  and  compound  tasks,  in¬ 
troducing  additional  constraints  between  tasks  not  found  in  task  networks  designed  for 
single  agent  planning.  This  shortcoming  has  been  addressed  by  task  networks  specifically 
tailored  to  multiagent  teams,  such  as  TAEMS  [16],  SharedPlans  [24],  and  team-oriented 
programs  [53]. 

Domain-independent  multiagent  infrastructures  such  as  GPGP  [37],  RETSINA  [60],  or 
Machinetta  [58]  provide  reusable  software  components  implementing  algorithms  to  ad¬ 
dress  problems  such  as  planning  and  scheduling,  subteam  formation,  and  task  and  resource 
allocation.  Planning  identifies  the  domain-level  tasks  the  team  will  execute  in  order  to 
achieve  high  level  goals,  based  on  the  requirements  for  task  execution  and  their  effects 
represented  in  the  task  networks.  Most  tasks  require  only  a  small  fraction  the  agents  and 
resources  of  the  full  team,  and  so  the  agents  are  organized  into  subteams  that  coordinate 
closely.  Task  allocation  is  the  problem  of  choosing  the  specific  tasks  for  agents  to  execute. 
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Flow  allocation  has  a  great  deal  of  overlap  with  these  problems.  The  plan  determines  the 
sink  nodes  on  which  flow  allocation  depends.  Conversely,  knowing  how  or  even  if  task 
requirements  can  be  met,  for  example  if  data  from  sensor  nodes  can  be  transmitted  to  a 
base  station  for  fusion,  is  crucial  for  planning.  Subteam  formation  affects  the  choice  of 
source  nodes  while  task  allocation  affects  which  sink  nodes  the  flows  should  go;  it  is  clear 
that  the  choice  and  quality  of  flows  through  the  environment  will  depend  on  the  both  of 
these  factors.  However,  this  overlap  with  other  multiagent  problems  is  not  unique  to  flow 
allocation;  instead,  the  extent  to  which  flow  allocation  interacts  with  the  other  problems 
just  reflects  the  degree  to  which  those  problems  interact  with  each  other. 

While  there  is  no  universally  accepted  way  to  resolve  these  interdependencies,  in  this  the¬ 
sis  we  assume  that  flow  allocation  occurs  after  planning  and  subteam  formation.  As  a 
consequence,  the  set  of  source  nodes  and  sink  nodes  are  known,  and  the  source  nodes  are 
partitioned  into  disjoint  groups  corresponding  to  the  subteams  partitioning  the  agents.  All 
source  nodes  in  a  group  must  send  flow  to  the  same  sink.  This  corresponds  to  the  subteam 
being  assigned  a  single  task,  which  may  be  a  compound  task  for  a  subteam  with  multiple 
agents.  For  example,  in  a  disaster  response  domain  [35],  a  subteam  may  be  composed  of 
several  fire  trucks  and  ambulances  assigned  the  task  of  responding  to  a  burning  building. 
This  compound  task  in  turn  has  component  tasks  assigned  to  each  individual  fire  truck  or 
ambulance  to  put  out  the  fire  and  to  rescue  survivors.  From  the  flow  allocation  perspec¬ 
tive  it  is  sufficient  to  consider  the  compound  task  that  involves  a  single  location  for  task 
execution  of  all  members  of  the  subteam. 

We  do  allow  for  overlap  with  a  limited  form  of  task  allocation.  In  many  of  the  problems 
we  consider  the  flow  allocation  problem  includes  choosing  a  sink  for  each  group  as  well 
as  choosing  the  flow  from  the  members  of  the  group  to  that  sink.  This  corresponds  to 
choosing  a  task  execution  location  for  each  group.  With  physical  tasks,  sinks  represents  the 
location  of  the  task  in  the  physical  environment,  like  the  burning  building  in  the  stricken 
city;  agents  must  physically  move  through  the  environment  to  reach  the  task  location.  Thus 
choosing  a  sink  represents  assigning  the  task  of  responding  to  a  particular  burning  building 
to  the  subteam.  With  computational  tasks,  the  source  nodes  represent  the  locations  of  input 
data,  while  the  sinks  represent  locations  where  the  computation  can  be  performed,  and  the 
flows  correspond  to  streams  of  data  through  the  communication  network.  For  example, 
any  base  station  could  perform  the  data  fusion  task  for  a  group  of  sensor  nodes  provided  it 
receives  all  of  the  sensor  readings.  Selecting  a  sink  corresponds  to  selecting  a  base  station 
to  perform  the  data  fusion. 

In  addition  to  facilitating  execution  of  domain-level  tasks,  flow  allocation  may  also  be 
applicable  to  control-level  computational  tasks  that  arise  from  the  team’s  coordination  al¬ 
gorithms.  A  number  of  multiagent  coordination  algorithms  require  groups  of  agents  to 
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transmit  data  to  a  single  location  for  computation.  Examples  of  such  control-level  compu¬ 
tational  tasks  include  winner  selection  for  auction-based  algorithms  [22],  partial  central¬ 
ization  in  distributed  constraint  optimization  [41,  51],  and  automated  monitoring  of  team 
plan  execution  [57].  Flow  allocation  can  be  used  to  optimize  the  choice  of  auctioneer, 
centralization  point,  or  plan  monitor. 


1.2  Flow  Allocation  in  Capacitated  Environments 

We  first  address  the  problem  of  flow  allocation  for  task  execution  in  capacitated  environ¬ 
ments.  These  are  most  natural  for  computational  tasks  in  communication  networks  where 
the  amount  of  data  that  can  be  transmitted  is  limited,  but  can  also  model  physical  environ¬ 
ments  with  transportation  limitations. 

The  environment  is  modeled  as  a  directed  graph  with  edge  capacities,  sink  nodes  represent 
task  execution  locations,  and  groups  of  source  nodes  represent  subteams  of  agents.  The 
source  nodes  of  each  group  have  an  amount  of  flow  that  they  must  transmit  to  the  sinks,  and 
the  flow  for  all  members  of  the  same  group  must  go  to  the  same  sink.  This  corresponds  to 
all  of  the  required  data  for  a  computational  task  being  sent  to  the  same  node  for  processing. 
The  total  amount  of  flow  on  each  edge  is  subject  to  the  capacity  limit  for  that  edge,  and  it 
may  not  possible  for  all  groups  to  transmit  their  full  amounts  of  flow  to  sinks.  The  flow 
allocation  problem  we  address  is  to  maximize  the  number  of  groups  able  to  transmit  their 
full  amounts  of  flow.  This  corresponds  to  enabling  the  maximum  number  of  subteams  to 
execute  their  assigned  tasks,  a  problem  we  formalize  as  the  Maximum  Satisfied  Groups 
(MaxSG)  problem. 

Many  well-known  flow  problems  such  as  maximum  flow  or  minimum  cost  flow  can  be 
solved  efficiently  by  exploiting  the  divisibility  of  flows  among  multiple  paths  to  meet 
capacity  constraints.  However,  we  show  that  MaxSG  is  intractable  even  when  flows  are 
divisible.  The  reason  for  this  are  two  integral  constraints.  First,  the  source  nodes  in  a 
group  must  all  send  their  flow  to  the  same  sink.  Choosing  a  sink  for  each  group  is  part 
of  the  MaxSG  problem,  and  it  corresponds  to  selecting  a  single  processing  node  on  which 
to  perform  a  computational  task  for  a  subteam  of  agents.  Second,  all  of  the  flow  from  the 
source  nodes  of  a  group  must  be  available  at  the  selected  sink  in  order  for  task  execution 
to  occur.  We  provide  an  algorithm  for  solving  MaxSG  optimally  by  expressing  these 
constraints  in  a  mixed  integer  linear  program  (MIFP). 

We  also  consider  two  variations  the  basic  flow  allocation  problem.  In  the  first,  we  restrict 
the  graph  of  the  environment  to  be  a  directed,  rooted  tree  with  a  single  sink  at  the  root.  This 
kind  of  functional  topology  arises  from  several  proposed  algorithms  designed  to  simplify 
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or  improve  the  efficiency  of  ad  hoc  wireless  networks.  We  provide  a  pseudo-polynomial 
time  algorithm  (the  running  time  depends  polynomially  on  the  magnitude  of  the  largest 
capacity)  for  optimally  solving  MaxSG  in  such  an  environment. 

In  the  second  variation  we  require  that  the  flows  be  unsplittable;  that  is,  flow  from  a  source 
node  cannot  be  divided  on  multiple  paths  to  the  sink.  This  is  useful  for  modeling  physical 
agents  like  robots  that  may  have  a  fixed  size  that  restricts  their  ability  to  move  through  the 
environment,  and  cannot  divide  themselves  into  multiple  smaller  parts  when  confronted 
with  a  bottleneck.  We  show  that  MaxSG  for  unsplittable  flows  is  also  intractable,  and 
moreover  we  prove  that  there  is  not  a  non-trivial  guaranteed  approximation  algorithm  un¬ 
less  P=NP. 


1.3  Network  Augmentation 

The  agents’  ability  to  modify  the  environment  is  motivated  by  approaches  to  augment 
communication  networks  by  the  addition  of  supplemental  nodes  to  improve  connectivity 
and  capacity  [68,  38,  61].  Establishing  and  maintaining  effective  wireless  communication 
networks  is  a  major  challenge  for  many  multiagent  domains,  such  as  future  military  oper¬ 
ations  and  disaster  response  teams.  In  these  applications,  agents  often  do  not  have  access 
to  established  communication  infrastructure,  and  so  must  rely  on  an  ad  hoc  network  to 
meet  their  communication  needs.  Limited  communication  ranges,  communication  capac¬ 
ity  constraints,  signal  obstruction  (by  terrain,  foliage,  and  buildings),  agent  mobility,  and 
wireless  interference  restrict  the  effectiveness  of  ad  hoc  networks  in  such  settings.  Aug¬ 
menting  the  network  through  the  addition  of  additional  nodes  is  one  way  to  address  these 
issues. 

These  supplemental  nodes  will  often  be  large  enough  and  have  sufficient  power  resources 
that  they  can  be  equipped  with  sophisticated  communication  hardware  with  longer  range 
and  higher  capacity  than  those  carried  by  people  on  the  ground  or  in  low-power  robots  and 
sensors.  For  example,  unmanned  aerial  vehicles  (UAVs)  and  unmanned  ground  vehicles 
(UGVs)  can  be  used  for  outdoor  environments,  small  robots  can  be  used  for  indoor  set¬ 
tings,  and  additional  base  stations  can  be  deployed  in  sensor  networks,  all  with  dedicated 
hardware  with  superior  capabilities  to  the  other  nodes  in  the  network.  UAVs  in  particular 
have  received  attention  because  they  possess  additional  advantages  such  as  operating  in 
largely  obstruction-free  airspace,  which  increases  range,  capacity,  and  reliability  of  com¬ 
munication  among  UAVs,  often  having  clearer  lines  of  sight  between  UAVs  and  agents  on 
the  ground,  and  being  able  to  travel  more  quickly  and  freely  than  ground-  based  agents, 
which  allow  them  to  respond  to  agents’  movements. 
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The  central  problem  of  network  augmentation  is  determining  where  the  supplemental 
nodes  should  be  positioned.  The  supplemental  nodes  can  be  used  in  a  variety  of  ways, 
for  instance  by  providing  connectivity  between  two  disconnected  parts  of  the  network,  im¬ 
proving  capacity  in  parts  of  the  network  with  heavy  traffic,  and  even  acting  as  processing 
nodes  for  computational  tasks,  due  to  their  superior  capabilities.  The  optimal  positioning 
of  supplemental  nodes  will  therefore  depend  on  the  communication  needs  of  the  network. 
At  the  same  time,  the  communication  needs  will  be  affected  by  the  positioning,  as  new 
flows  may  become  possible  and  sink  locations  change.  Network  augmentation  thus  adds 
an  additional  and  interdependent  level  of  optimization  to  the  flow  allocation  problem. 

Network  augmentation  therefore  involves  partial  knowledge  of  both  the  topology  and  the 
communication  needs.  In  considering  flow  allocation,  we  assumed  a  known  topology  but 
partial  knowledge  of  the  communication  needs:  we  knew  the  source  groups  but  not  the 
sink  assigned  to  each  source  group.  Other  work  in  network  design  typically  assumes  full 
knowledge  of  the  communication  needs  in  the  form  of  flow  demands  for  specific  source- 
sink  pairs,  but  only  partial  knowledge  of  the  topology.  In  addressing  network  augmentation 
for  flow  allocation  we  must  simultaneously  address  both  forms  of  partial  knowledge  to  find 
optimal  solutions. 

We  represent  the  initial  network  without  supplemental  nodes  as  a  graph  with  capacity 
constraints  on  edges.  This  can  represent  the  effects  of  obstructions,  interference,  signal 
attenuation,  and  other  factors  in  the  properties  of  the  network.  In  addition,  we  restrict  the 
deployment  of  supplemental  nodes  to  a  finite  set  of  possible  locations  whose  connectivity 
properties  are  known,  which  is  represented  by  the  potential  network.  These  locations 
may  be  pre-selected  as  desirable  or  acceptable,  or  generated  as  a  finite  approximation.  In 
practice,  these  graphs  can  be  generated  in  real-time  if  the  relevant  properties  of  the  wireless 
network  are  known. 

Flow  allocation  for  source  groups  is  performed  on  the  actual  network  determined  by  a 
deployment  of  supplemental  nodes  to  possible  locations  in  the  potential  network,  where 
possible  locations  (and  incident  edges)  in  the  potential  network  that  have  not  been  filled  by 
a  supplemental  node  are  removed.  The  supplemental  nodes  are  able  to  act  as  sinks  as  well 
as  relays,  and  we  consider  two  related  problems,  the  first  of  maximizing  the  number  of 
satisfied  groups  given  a  fixed  number  of  supplemental  nodes  and  the  second  of  finding  the 
minimum  number  of  supplemental  nodes  required  for  all  groups  to  be  able  to  be  satisfied. 
We  prove  that  these  problems  are  intractable  and  provide  optimal  and  heuristic  algorithms 
for  solving  them. 
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1.4  Flow  Allocation  in  Adversarial  Environments 


Some  of  the  most  common  multiagent  domains,  those  in  military  and  law  enforcement 
settings,  are  inherently  hostile.  The  agents  must  operate  in  an  environment  that  is  not  just 
the  result  of  neutral  processes  but  is  influenced  by  a  purposive  adversary  whose  interests 
are  opposed  to  those  of  the  agents.  This  creates  a  very  different  setting  for  flow  allocation 
than  a  neutral  environment  or  one  that  can  be  affected  through  network  augmentation  by 
the  agents.  In  those  contexts,  the  problem  is  one  of  straightforward  if  computationally 
difficult  optimization.  In  adversarial  settings,  flows  must  be  chosen  strategically  even  as 
the  adversary  is  actively  and  rationally  attempting  to  counter  their  efforts. 

We  consider  costly  environments  where  there  are  no  capacity  limitations,  but  transmitting 
flow  on  each  edge  incurs  a  cost  for  the  agents  that  is  proportional  to  the  amount  of  flow 
sent.  This  can  be  represented  by  defining  a  cost  vector  over  the  edges  of  the  graph  and 
computing  the  total  costs  as  the  scalar  product  of  the  cost  vector  and  the  flow  vector  de¬ 
scribing  the  amount  of  flow  sent  on  each  edge.  This  setting  is  widely  applicable  to  both 
physical  environments  and  communication  environments.  For  example,  costs  may  reflect 
the  time  it  takes  for  a  vehicle  to  move  through  a  city,  or  battery  depletion  used  to  transmit 
across  a  communication  link.  The  agent  team  seeks  to  minimize  the  total  costs  required  to 
execute  their  tasks. 

In  the  absence  of  an  adversary,  it  is  straightforward  to  compute  satisfying  flows  for  the 
source  groups  that  minimize  the  total  costs  incurred  by  the  agents,  and  this  can  be  done 
in  polynomial  time.  In  an  adversarial  environment,  the  adversary  can  carry  out  a  num¬ 
ber  of  attacks,  each  of  which  adds  special  costs  called  harm  to  the  graph,  equivalent  to 
adding  a  new  cost  vector  to  the  graph.  For  example,  with  one  attack  the  adversary  may 
jam  a  specific  region  of  the  communication  network,  forcing  nearby  nodes  to  use  more 
battery  power  due  to  higher  transmission  power  or  a  greater  number  of  retransmissions. 
In  a  physical  environment,  the  adversary  may  place  an  obstacle  on  a  road  that  slows  the 
movement  of  vehicles  over  that  road.  When  the  adversary  carries  out  multiple  attacks,  the 
cost  vectors  for  each  attack  are  summed  and  used  to  compute  the  total  harm  suffered  by 
the  agents  in  addition  to  the  neutral  environmental  costs. 

This  interaction  is  naturally  modeled  using  the  theoretical  framework  of  game  theory.  The 
harm  depends  on  both  the  flows  used  by  the  agents  and  the  attacks  carried  out  by  the 
adversary,  and  so  the  agents  must  reason  about  what  attacks  the  adversary  will  carry  out, 
even  as  the  adversary  contemplates  what  flows  the  agents  will  use.  Assuming  that  both 
sides  are  rational,  self-interested  actors,  game  theory  provides  a  rigorous  framework  for 
reasoning  strategically  about  their  behaviors.  In  particular,  the  game  theoretic  concept  of 
Nash  equilibrium  is  a  solution  in  which  both  sides  are  playing  optimally  given  the  behavior 
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of  the  other  side. 


We  model  flow  allocation  in  adversarial  environments  as  a  two-player  game  played  on  a 
directed  graph.  One  player,  the  sender,  represents  the  team  of  agents  and  chooses  satis¬ 
fying  flows  through  the  network  for  the  source  groups.  The  other  player,  the  adversary, 
chooses  a  set  of  attacks  from  a  set  of  possible  attacks.  The  players  choose  their  strategies 
without  being  able  to  observe  the  strategy  chosen  by  the  other  player,  but  they  are  able  to 
reason  about  what  the  other  player  would  do  based  on  common  knowledge  about  the  pay¬ 
offs.  We  consider  both  zero-sum  games,  where  the  players  receive  payoffs  that  are  exactly 
opposite  (based  on  the  harm  suffered  by  the  agents),  and  non-zero-sum  games  where  the 
payoff  to  the  sender  is  based  on  the  harm  and  the  environmental  costs,  while  the  payoff  to 
the  adversary  is  based  on  the  harm  and  costs  of  playing  each  attack. 

These  games  are  challenging  to  solve  using  traditional  game  theory  because  the  players 
have  very  large  strategy  spaces:  the  continuous  space  of  all  satisfying  flows  for  the  sender, 
and  the  exponential  space  of  subsets  of  possible  attacks  based  on  the  maximum  number 
of  attacks  that  can  be  carried  out  simultaneously.  However,  we  show  how  the  structure 
of  the  games  can  be  used  to  efficiently  find  equilibrium  solutions  in  polynomial  time.  We 
also  show  how  various  types  of  uncertainty  about  payoffs  and  adversary  capabilities  can 
be  modeled  as  Bayesian  games  and  extend  our  algorithm  to  solving  these. 

Finally,  we  consider  games  where  the  sender  must  commit  to  a  strategy  that  can  be  ob¬ 
served  by  the  adversary.  Although  the  Nash  equilibrium  strategies  can  be  used,  we  show 
that  this  may  lead  to  very  bad  payoffs  for  the  sender  due  to  indifference  on  the  part  of  the 
adversary.  In  a  security  setting,  the  sender  cannot  leave  the  difference  between  success 
and  disaster  up  to  the  whims  of  the  adversary.  Instead,  we  show  how  the  sender  can  devi¬ 
ate  by  an  arbitrarily  small  amount  from  an  equilibrium  strategy  in  order  to  induce  even  a 
worst-case  adversary  to  play  a  strategy  that  results  in  the  most  favorable  expected  payoff 
for  the  sender. 


1.5  Contributions 

This  thesis  makes  the  following  contributions: 

•  a  formulation  of  the  flow  allocation  problem  in  capacitated  environments  and  a  proof 
that  it  is  strongly  NP-hard  (Chapter  2); 

•  a  pseudo-polynomial  time  algorithm  for  flow  allocation  on  trees  (Chapter  2); 

•  a  proof  that  flow  allocation  on  general  graphs  where  flows  cannot  be  split  is  strongly 
NP-hard  to  solve  optimally  and  cannot  be  meaningfully  approximated  in  polynomial 
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time  unless  P  =  NP  (Chapter  2); 

•  a  proof  that  network  augmentation  is  NP-hard  to  solve  optimally,  and  algorithms  for 
solving  it  heuristically  (Chapter  3); 

•  a  linear  programming  formulation  for  finding  equilibrium  strategies  in  zero-sum 
flow  allocation  games  (Chapter  4); 

•  a  linear  programming  formulation  for  finding  equilibrium  strategies  in  non-zero- sum 
flow  allocation  games  with  costs  (Chapter  5); 

•  a  new  equilibrium  refinement,  inducible  Stackelberg  equilibrium,  for  non-zero-sum 
flow  allocation  games  with  commitment,  and  an  algorithm  for  computing  the  opti¬ 
mal  inducible  Stackelberg  equilibrium  (Chapter  5). 
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Chapter  2 

Flow  Allocation  in  Capacitated 
Environments 


We  begin  by  considering  the  problem  of  allocating  flows  in  a  capacitated  environment. 
Communication  networks  are  natural  examples  of  these  kinds  of  environments,  as  many 
multiagent  systems  use  wireless  networks  that  are  tightly  constrained  in  the  amount  of  data 
that  can  be  sent  between  agents.  Despite  these  limitations,  groups  of  agents  must  often 
transmit  streams  of  data  to  a  single  location  in  the  network  as  input  to  a  computational 
task.  For  example,  locating,  recognizing,  and  tracking  an  object  with  a  sensor  network 
requires  fusing  data  streams  from  multiple  sensor,  while  plan  monitoring  requires  access 
to  status  information  from  agents  in  the  subteam  executing  the  plan.  Because  of  capacity 
constraints,  it  may  not  be  possible  for  all  tasks  to  be  executed.  The  fundamental  flow 
allocation  problem  is  to  choose  the  task  locations  and  flows  so  that  the  maximum  number 
of  tasks  can  be  performed.  We  formalize  this  problem  next. 


2.1  Flow  Allocation 

In  this  section  we  formalize  the  flow  allocation  problem.  A  problem  instance  has  three 
parts:  a  graph,  sink  nodes,  and  groups  of  source  nodes.  These  correspond  to  the  environ¬ 
ment,  task  execution  locations,  and  agent  subteams. 

The  environment  is  represented  as  a  directed  graph  G  =  (V.  E )  where  V  is  the  set  of  nodes 
and  E  is  the  set  of  edges;  following  convention  we  refer  to  such  a  graph  as  a  network.  The 
number  of  nodes  and  edges  are  denoted  by  n  =  |Vj  and  m  =  \E\,  respectively.  Each 
edge  e  e  E  has  a  weight  w{e)  G  [0,  +oo]  representing  the  capacity  of  that  edge.  The 
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Symbol 

Description 

b(s),bs 

source  requirement  of  s  E  S 

E 

set  of  edges 

fl 

the  amount  of  flow  on  edge  e  E  E  for  group  St 

G  =  (V,  E) 

directed  graph  of  the  environment 

K 

number  of  groups 

m 

number  of  edges 

n 

number  of  nodes 

P(v) 

parent  of  node  v 

S  c  V 

set  of  source  nodes 

&  =  {SU...,SK} 

set  of  groups  partitioning  S 

t  eV\S 

sink  node 

Tr  =  (V,  E,  t,  w ) 

Tree-In  input  tree  with  nodes  E,  edges  E,  root  t  e  V,  and  edge 
capacities  w 

V 

set  of  nodes 

w(e) 

capacity  of  edge  e  E  E 

Table  2.1:  Summary  of  notation  for  Chapter  2 


capacity  of  an  edge  may  either  be  a  non-negative  real  number  (in  which  case  the  edge  has 
limited  capacity),  or  positive  infinity  (in  which  case  the  edge  has  unlimited  capacity).  As 
we  will  see  later  the  inclusion  of  positive  infinity  as  a  value  for  w(e)  is  convenient  but  not 
necessary  as  any  in  any  instance  a  suitable  large  finite  value  could  be  substituted  instead. 

The  set  of  sink  nodes  T  C  V  \  S  represent  the  possible  locations  of  task  execution  in  the 
environment.  With  physical  tasks,  each  sink  represents  the  location  of  a  task  in  the  physical 
environment.  For  example,  in  the  disaster  response  domain  a  sink  may  represent  a  burning 
building.  With  computational  tasks,  each  group  is  assumed  to  have  an  implicit  task,  and  the 
sinks  represent  processing  nodes  where  the  computation  can  be  performed.  For  example, 
in  a  distributed  sensor  network  each  group  corresponds  to  a  set  of  sensor  nodes  that  require 
data  fusion  of  their  sensor  readings,  and  sinks  correspond  to  base  stations  where  this  fusion 
can  occur.  We  assume  without  loss  of  generality  that  sink  nodes  have  no  outgoing  edges; 
if  there  is  a  sink  node  t  with  outgoing  edges  we  can  add  a  new  sink  node  t' ,  add  an  edge 
from  t  to  t'  with  unlimited  capacity,  and  remove  t  from  T.  An  example  of  this  process  is 
shown  in  Figure  2.1. 

The  source  nodes  S  C  V  represent  the  agents  in  the  environment.  The  source  nodes  are 
partitioned  into  K  groups  denoted  &  =  {Si, . . . ,  SK}  reflecting  the  organization  of  the 
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Figure  2.1:  Transforming  a  network  with  outgoing  edges  from  a  sink  node  to  one  without 
outgoing  edges  from  that  sink  node.  A  new  sink  node  is  added  with  an  edge  with  unlimited 
capacity  from  the  old  sink  node.  This  adds  at  most  \T\  additional  nodes  and  edges. 

agents  into  non-overlapping  subteams,  with  each  source  node  in  exactly  one  group.  The 
group  index  group(s)  of  a  source  node  s  G  S  is  the  index  i  such  that  s  G  S}. 

The  flow  requirement  is  a  function  b  :  S  — >■  |0,  +oo)  mapping  source  nodes  to  the  amount 
of  flow  that  must  be  sent  from  that  source  node  to  a  sink.  Because  all  source  nodes  in 
a  group  must  send  their  flow  to  the  same  sink,  it  is  often  convenient  to  index  the  flow 
requirements  both  by  the  source  node  and  the  source  group,  denoted  by  b\  =  b(s)  for 
s  G  Si.  When  convenient  we  may  extend  the  domain  of  b  to  all  nodes,  not  just  those  in  S, 
with  the  understanding  that  b(v)  =  0  for  all  v  ^  S  and  similarly  blv  =  0  for  all  v  ^  S,;. 

A  solution  to  the  flow  allocation  problem  has  three  components:  a  set  of  groups  for  which 
flow  has  been  allocated,  the  assignment  of  sinks  to  these  groups,  and  the  flows  that  have 
been  allocated.  We  let  X  C  &  denote  the  set  of  groups  for  which  flow  has  been  allocated. 
For  each  Si  E  X  we  let  x,  G  T  denote  the  sink  assigned  to  group  St. 

A  multicommodity  network  flow  for  X  is  a  function  /  :  E  — *  [0,  +oc};i  that  maps  edges 
to  a  non- negative  amount  of  flow  on  that  edge  for  each  group  in  X.  For  convenience  we 
usually  denote  the  flow  for  the  Ah  group  on  an  edge  e  =  (u,  v)  G  E  by  f\  or  f'fl,  and  in 
cases  where  there  is  a  single  group  (i.e.,  &  =  {S'})  we  sometimes  omit  the  superscript.  A 
flow  must  satisfy  the  flow  capacity  and  flow  conservation  constraints: 


VeGF 


(flow  capacity)  (2.1) 


Si&X 


E  A=  E  fuV  Wv  G  V  \T,  Si  G  X  (flow  conservation)  (2.2) 


(u,v)ee 


(v,u)eE 


The  capacity  requirement  constraint  in  Equation  (2.1)  requires  that  the  total  flow  for  all 
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Figure  2.2:  MaxSG-Dec  is  strongly  NP-complete:  Reduction  from  the  Bin  Packing  deci¬ 
sion  problem. 


groups  on  each  edge  must  be  no  greater  than  the  capacity  of  that  edge.  The  flow  conser¬ 
vation  constraint  in  Equation  (2.2)  requires  that  the  amount  of  flow  of  each  type  that  exits 
a  node  is  equal  to  the  sum  of  the  amount  of  flow  of  that  type  originating  at  that  node  (i.e., 
the  flow  requirement)  and  the  amount  of  flow  of  that  type  entering  the  node,  for  all  nodes 
other  than  the  sink.  From  these  two  constraints  it  is  clear  that  no  edge  can  receive  a  total 
amount  of  flow  that  exceeds  B  =  'ffses  bs,  and  so  the  weights  w(e)  could  be  limited  to 
the  interval  [0,  B}. 

The  flow  for  each  group  S)  €  X  should  be  sent  to  the  sink  Xi  assigned  to  it.  This  is 
expressed  in  the  sink  flow  constraint: 

=  v si  e  X  (sink  flow)  (2.3) 

(u,xi)eE  sESi 

If  a  flow  satisfies  the  sink  flow  constraint  in  Equation  (2.3)  we  say  that  the  flow  is  a  sat¬ 
isfying  flow  for  X  and  that  the  groups  in  X  are  satisfied.  Satisfied  groups  correspond  to 
agent  subteams  that  are  able  to  proceed  with  task  execution.  A  natural  problem  is  to  en¬ 
able  as  many  subteams  as  possible  to  execute  tasks.  This  is  the  problem  of  maximizing 
the  number  of  satisfied  groups,  MaxSG. 

Problem  1.  Maximum  Satisfied  Groups  (MaxSG).  Given  graph  G  =  (V,  E)  with  capac¬ 
ities  w,  source  groups  (5,  source  requirements  b,  and  sinks  T,  find  X  C6,  sink  assignment 
x  and  satisfying  flow  /  for  X  such  that  \X\  is  maximized. 

The  decision  version  of  this  problem,  MaxSG-Dec,  is  to  decide  whether  it  is  possible  to 
allocate  flows  to  satisfy  at  least  k  groups. 

Theorem  1.  MaxSG-Dec  is  strongly  NP-complete. 
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Proof.  To  show  that  MaxSG-Dec  is  in  NP  we  observe  we  can  verify  in  polynomial  time  if 
\X\  >  k  and  /  is  a  satisfying  flow  for  X . 

We  prove  NP-hardness  by  reduction  from  Bin  Packing,  which  is  known  to  be  strongly 
NP-complete  [21].  The  Bin  Packing  decision  problem  is  the  NP-complete  problem  of 
deciding  if  a  set  of  nBp  items  of  different  size  can  be  placed  into  kBp  bins.  Formally, 
it  is  the  problem  of  deciding  if  a  set  of  numbers  {ai, . . . ,  angp}  with  a*  G  (0, 1]  can  be 
partitioned  into  kBp  sets  Birii, . . . ,  BinkBP  such  that  ^agBin.  <  1  for  1  <  i  <  kBP. 

From  a  Bin  Packing  instance  we  construct  an  instance  of  MaxSG-Dec,  as  illustrated  in 
Figure  2.2.  First,  create  nBP  source  nodes  su . . . ,  sngp  and  set  the  flow  requirements 
bi  =  ai.  Place  each  source  node  in  its  own  group.  Next  create  kBp  nodes  Vi, . . . ,  vkBP  and 
add  edges  from  each  source  node  to  each  of  these  newly  created  nodes.  Set  the  capacity 
of  these  edges  to  positive  infinity.  Add  kBp  sink  nodes,  ti, ... ,  tkBP  and  add  an  edge  with 
unit  capacity  between  v%  and  ti  for  1  <  i  <  kBP.  Finally  set  k  —  nBP  so  that  the  decision 
is  whether  it  is  possible  to  allocate  flow  for  all  of  the  groups. 

Suppose  that  there  is  a  flow  satisfying  all  nBP  groups.  Because  each  group  is  a  singleton, 
this  means  that  the  flow  from  each  source  node  can  be  sent  to  sinks.  Let  Yt  C  S  be  the  set 
of  all  source  nodes  sending  flow  to  sink  ti,  that  is,  Y,  =  { G  S  :  Xj  =  U}.  From  the  flow 
capacity  constraint  for  edge  (vl,  ti),  it  follows  that 

aj  <1  Vi  G  [l..kBp\ 

Sj 


Because  all  groups  are  satisfied  and  each  group  is  a  singleton,  it  follows  that  the  {Y,} 
partition  S  into  at  most  kBP  sets.  Thus  there  is  a  bin  packing  using  at  most  kBP  bins. 

Conversely,  suppose  that  there  is  a  bin  packing  using  at  most  kBP  bins.  For  each  aj  G  Birii 
we  set  Xj  =  ti  and  f3s  =  /*.t.  =  a3  (and  0  for  other  edges).  By  the  bin  packing 
constraints  we  know  that  the  flow  capacity  constraints  on  the  edges  (ry  ,  U )  must  be  satisfied 
and  hence  all  nBp  groups  are  satisfied. 

The  answer  to  a  Bin  Packing  instance  is  YES  if  and  only  if  the  answer  to  the  MaxSG-Dec 
instance  is  YES,  so  MaxSG  is  strongly  NP-hard,  and  therefore  strongly  NP-complete.  □ 

We  solve  MaxSG  by  formulating  it  as  a  mixed  integer  linear  program  (MILP)  and  using 
existing  algorithms  for  solving  MILPs  to  compute  a  solution.  Solving  a  MILP  is  known 
to  be  NP-hard,  but  this  is  acceptable  given  Theorem  1.  Our  formulation  for  MaxSG  is 
MILP  1.  We  must  specify  the  flow  capacity,  flow  conservation,  and  sink  flow  constraints 
as  a  set  of  constraints  that  are  linear  in  the  variables  being  solved  for.  At  first  glance,  it 
may  appear  that  Equations  (2.1)  -  (2.3)  are  of  this  form  as  they  are  linear  in  terms  of  /. 
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MILP  1  MILP  formulation  of  MaxSG. 


K 


Maximize  Xlt 

*’X  i= 1  t£T 

subject  to: 

i>:EA'«+  E  /»»  = 

V  ri 

/  J  J  uv 

Mv  e  V\T,i  e  [1..K] 

(2.4) 

(2.5) 

teT  (u,v)£E 

K 

/e  <  w{e) 

i=  1 

(v,u)eE 

Ve  G  E 

(2.6) 

v  fit=xuj2’’i 

Vt  eT,i  e  [1..K] 

(2.7) 

(u,t)eE  s€Si 

^2xit<i 

Vi  e  [l..K\ 

(2.8) 

t£T 

Xit  =  o 

Vi  e  [l..K],te  T 

(2.9) 

/:>  o 

Me  e  E,i  e  [l..K\ 

(2.10) 

Xu  £  {0, 1} 

Vi  e  [i..K],te  T 

(2.11) 
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However,  the  output  X  is  also  used  in  all  three:  to  determine  the  sum  in  the  flow  capacity 
constraint,  and  to  determine  the  presence  or  absence  of  constraints  for  different  values  of  i 
in  the  flow  conservation  and  sink  flow  constraints.  We  linearize  these  constraints  through 
the  addition  of  integer  indicator  variables  Xlt  for  1  <  i  <  K  and  t  e  T.  Each  variable 
can  take  the  value  1  or  0,  with  Xlt  =  1  if  t  is  the  sink  assigned  to  St  (that  is,  if  xt  =  t ) 
and  Xlt  =  0  otherwise.  Because  each  group  is  assigned  at  most  one  sink,  it  follows  that  at 
most  one  Xlt  for  each  i  is  non-zero.  This  is  expressed  by  the  constraint  in  Equation  (2.8). 
If  the  sum  equals  0,  then  no  sink  has  been  assigned  to  group  St  and  so  St  X.  In  this 
way  the  integer  collectively  represent  X  in  addition  to  the  sink  assignment  x.  As  a  result, 
the  objective  of  maximizing  \X\  is  equivalent  to  maximizing  the  sum  of  the  Xit  variables, 
which  is  the  objective  of  MILP  1  as  shown  by  Equation  (2.4). 

Flow  conservation  is  expressed  as  Equation  (2.5),  which  requires  that  flow  originates  at  v 
for  group  Si  if  and  only  if  a  sink  has  been  assigned  to  St.  As  a  result,  for  all  e  G  77,  it 
follows  that  fl>  0  if  and  only  if  a  sink  has  been  assigned  to  St,  and  so  we  can  express  the 
flow  capacity  constraint  by  summing  over  all  possible  groups  as  shown  in  Equation  (2.6), 
knowing  that  there  will  only  be  a  positive  amount  of  flow  for  groups  in  X.  Finally,  the 
sink  flow  constraint  is  captured  by  Equation  (2.7),  which  requires  that  the  total  amount  of 
flow  for  group  St  entering  a  sink  t  must  be  J2seS.  K  if  t  is  the  sink  assigned  to  S)  and  0 
otherwise. 


2.1.1  Alternative  Capacity  Models 

We  consider  flows  that  are  limited  by  capacities  on  the  edges.  In  this  subsection  we  de¬ 
scribe  how  to  accommodate  some  other  types  of  capacities. 


Node  Capacities 

With  node  capacities  the  amount  of  flow  that  can  pass  through  each  node  v  e  V  is  limited 
to  a  maximum  amount  w(v)  >  0.  In  communication  networks  this  arises  when  the  ability 
of  a  node  to  process  communication  is  limited.  One  way  this  can  arise  is  when  the  team 
includes  humans  who  must  manually  relay  the  information. 

A  graph  G  =  (V,  E)  with  node  capacities  can  be  transformed  into  a  new  graph  G'  = 
(V',E')  with  an  additional  \V\  nodes  and  edges.  This  construction  is  illustrated  in  Fig¬ 
ure  2.3.  For  each  node  v  G  V  we  add  two  nodes  vin  and  vout  to  V,  and  an  edge  (vin,  vout) 
to  E'  with  edge  capacity  w(vin,vout)  =  w(v).  This  edge  represents  the  limitation  in  the 
node  u’s  ability  to  process  flow.  To  preserve  the  overall  topology  of  the  original  network, 
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Figure  2.3:  Transforming  a  network  with  node  capacities  to  one  with  edge  capacities, 
for  each  edge  (u,  v )  G  E  we  add  an  edge  (uout,  vin)  to  E' . 

In  the  MILP  formulation  we  can  more  easily  incorporate  node  capacities  by  adding  a  new 
constraint: 


^  fuv  <  w(v)  Mv  G  V  (node  capacities). 

(u,v)£E 


Wireless  Interference 


Many  multiagent  teams  utilize  wireless  networks  that  share  the  same  channel  for  commu¬ 
nication.  In  such  cases,  individual  capacity  limitations  on  each  link  are  often  far  less  im¬ 
portant  than  the  effect  of  wireless  interference.  Most  nodes  use  omnidirectional  antennas 
and  the  transmissions  from  a  node  are  heard  by  all  nearby  nodes,  causing  interference.  A 
simple  way  to  model  this  to  a  first  approximation  in  the  MILP  formulation  uses  adjacency 
in  G  as  an  indication  of  which  nodes  can  interfere,  on  the  premise  that  if  the  transmis¬ 
sion  from  a  node  can  be  used  to  communicate  with  another  node,  it  can  also  interfere 
with  that  node.  More  sophisticated  approaches  may  use  a  neighbors-of-neighbors  model 
to  reflect  the  rule-of-thumb  that  the  interference  range  is  generally  twice  the  transmission 
range  [65],  or  conflict  graphs  that  explicitly  describing  which  links  interfere  [31]. 

We  assume  that  transmissions  use  a  shared  channel  and  the  edge  capacities  reflect  full 
utilization  of  that  channel  in  the  absence  of  other  transmissions.  When  a  node  v  G  V 
transmits  on  multiple  edges,  it  divides  time  transmitting  to  each  recipient,  with  the  fraction 
of  time  equal  to  fvu/w(v ,  u ).  This  leads  to  the  following  constraint  on  outgoing  flow: 


E 

(■ v,u)GE 


fv 


w(v,  u ) 


<  1 


\/v  G  V 


To  receive  a  transmission,  v  must  have  an  open  time  slot  to  receive  the  transmission. 
However,  due  to  interference  time  slots  are  filled  with  all  transmissions  from  neighbors 
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of  v,  not  just  those  intended  for  v.  We  first  consider  the  available  time  slots  without 
considering  interference.  We  add  a  binary  decision  variable  Zv  for  each  v  e  V  to  indicate 
whether  v  can  receive  flow.  If  Zv  =  1  then  flow  is  allowed  to  be  sent  to  i;;  if  Zv  —  0  then 
no  flow  can  be  sent  to  v.  Thus  we  get  the  constraint 

fuv  ,  <  Zv  W  e  V. 

w(u ,  V) 

Next  we  consider  the  effect  of  interference.  If  flow  is  sent  to  v,  the  total  time  slots  must 
be  used  between  received  transmissions  and  interference.  If  flow  is  not  sent  to  v  we  don’t 
care  how  much  time  slots  are  suffer  from  interference.  This  is  achieved  by  using  a  suitably 
large  constant  M  in  the  following  constraint: 

y  luv'  .  <  1  +  M(1  -  Zv)  Vv  e  V. 

w(u,  in 

(■ u,v)eE  (u,v')e E  v  ’  ' 


When  Zv  =  1,  the  right-hand  side  becomes  1,  so  at  most  all  of  the  time  slots  can  be  used. 
When  Zv  =  0  the  right-hand  side  becomes  M  +  1,  removing  the  limit  on  the  amount  of 
interference  for  suitably  large  M.  However,  v  cannot  receive  any  flow  due  to  the  previous 
constraint. 


2.2  Flow  Allocation  on  Trees 

We  turn  now  from  environments  modeled  as  general  directed  graphs  with  multiple  sinks 
to  environments  with  a  more  limited  topology  and  a  single  sink.  In  communication  net¬ 
works,  simple  network  structures,  and  trees  in  particular,  are  often  favored  for  reasons  of 
efficiency  and  scalability.  Trees  greatly  simplify  routing,  while  also  arising  naturally  as  a 
result  of  power-conserving  mechanisms  such  as  hierarchical  routing  [28,  29],  data  aggre¬ 
gation  [1],  and  topology  control  [49].  If  flow  allocation  for  computational  tasks  is  done  in 
these  kinds  of  networks,  the  functional  network  topology  used  as  input  will  be  a  tree. 

In  this  section  we  consider  a  special  case,  the  Tree-In  case,  in  which  G  —  Tr  —  ( V. ]  E,  t ) 
is  a  directed  tree  with  a  single  sink  t  at  the  root  and  all  edges  directed  toward  the  root. 
In  addition,  we  assume  that  the  source  nodes  are  each  in  their  own  group,  so  that  ©  = 
{{s}  :  s  G  S'}.  Because  of  this  assumption,  in  this  section  we  find  the  set  of  source  nodes 
S'  C  S  whose  flows  can  be  allocated,  with  the  understanding  that  this  is  equivalent  to 
finding  X  =  {{s}  :  s  e  S'}. 


19 


We  denote  the  subtree  rooted  at  a  node  v  (including  the  node  v  itself)  by  Trv  =  (Vv,  Ev). 
We  also  use  p(v)  to  denote  the  parent  of  any  nodes  v  G  V  other  than  the  root.  Without  loss 
of  generality  we  can  assume  that  there  are  no  leaf  nodes  in  Tr  that  are  not  source  nodes;  if 
there  are,  we  can  remove  them  without  affecting  the  problem  because  they  cannot  be  on  a 
flow  path  from  any  of  the  sources  to  the  sink.  Furthermore,  we  can  assume  that  all  source 
nodes  are  leaves  in  Tr;  for  an  interior  source  node  s,  we  can  add  a  new  leaf  node  s'  with 
edge  (s',  s ),  designate  s'  as  a  source  node  with  bs>  =  bs,  and  remove  s  from  S.  Together 
these  assumptions  mean  that  we  can  assume  a  node  is  a  leaf  if  and  only  if  it  is  a  source 
node. 

The  key  characteristic  of  the  tree  topology  is  that  there  is  a  unique  path  from  v  to  the  root 
t  for  any  v  G  V.  Because  of  this,  the  flow  conservation  constraint  requires  that  all  of 
the  flow  that  originates  in  a  subtree  Trv  (for  v  ^  t)  must  be  transmitted  over  the  edge 
(v,  p(v)).  If  S'  is  the  set  of  source  nodes  whose  groups  are  satisfied,  it  follows  that  Vv  D  S' 
are  the  satisfied  source  nodes  in  the  subtree  rooted  at  v,  and  so  Yls'ev  ns1  bs>  is  the  total 
amount  of  flow  on  edge  (v,p(v)).  For  the  Tree-In  case,  the  flow  capacity  constraint  is 
equivalent  to  requiring  that  that  amount  of  flow  is  no  greater  than  w(v,p(v))  for  all  v  other 
than  the  root. 

The  following  formalizes  the  Tree-In  case  of  the  MaxSG  problem. 

Problem  2.  (Tree-In  case  of  MaxSG.)  Given  a  directed  tree  Tr  =  (1/,  E,  t )  with  root 
t  G  V,  integral  edge  capacities  w,  and  source  nodes  S  with  integral  source  requirements 
b,  find  S'  C  S  such  that 

1.  |  S'  |  is  maximized;  and 

2-  Ervins'  bs'  <  w(v,p(v))  for  all  v  G  V\{t}. 

The  first  condition  is  equivalent  to  maximizing  the  number  of  satisfied  source  groups  as 
each  source  node  is  in  its  own  group.  The  second  condition  is  equivalent  to  the  flow 
capacity  constraint,  as  described  above. 

Recall  from  the  proof  Theorem  1  that  a  Bin  Packing  instance  was  reduced  to  an  instance 
of  MaxSG  where  each  source  node  was  in  its  own  group.  Thus,  MaxSG  is  still  strongly 
NP-hard  even  when  restricted  to  source  groups  that  are  singletons,  and  so  there  are  no 
known  pseudopolynomial-time  algorithms  for  solving  these  instances.  A  natural  question 
is  whether  this  remains  true  in  the  Tree-In  case,  where  there  is  a  single  sink  and  the  graph 
is  a  tree. 

The  answer  is  no.  Algorithm  1  is  a  multi-layer  dynamic  programming  algorithm  that 
solves  the  Tree-In  case  optimally  in  pseudo-polynomial  time.  It  proceeds  by  computing 
two  functions,  one  for  the  value  of  an  optimal  solution  and  one  for  an  optimal  solution 
itself,  in  a  bottom-up  fashion.  The  first  function  is  Fv(y),  the  maximum  number  of  source 
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nodes  in  the  subtree  rooted  at  v  (including  v  itself)  that  can  have  their  flow  requirements 
satisfied  if  the  total  amount  of  flow  sent  to  v  is  at  most  y.  The  second  function  is  Sv(y),  a 
maximum  set  of  source  nodes  in  the  subtree  rooted  at  v  (including  v  itself)  that  can  have 
their  flow  requirements  satisfied  if  the  total  amount  of  flow  sent  to  v  is  at  most  y. 

For  each  v  G  V,  we  must  compute  Fv(y)  and  Sv (y)  for  values  of  y  G  [0 ..Bv\,  where 
Bv  =  ^2seSrTv  b(s),  the  sum  of  flow  requirements  in  the  subtree  rooted  at  v.  As  we 
move  up  the  tree,  we  compute  Fv(y)  and  Sv(y)  based  on  the  values  for  the  children  of  v. 
The  optimal  substructure  that  we  exploit  is  that  Fv(y)  is  determined  exactly  by  the  values 
Fu(z)  for  its  children  u,  and  that  we  can  compute  Fv(y)  sequentially  by  iterating  over  the 
children  of  v. 

We  begin  Algorithm  1  by  initializing  Bv  for  all  v  (line  1).  These  values  will  be  used  later 
in  the  algorithm.  In  line  2  we  then  order  the  nodes  according  to  their  depth-first  post  order. 
This  guarantees  that  all  children  of  a  node  v  will  be  processed  before  v  is  processed,  so 
that  we  can  recursively  use  the  F  values  for  the  children  in  computing  the  F  value  for  v. 
We  then  iterate  through  the  nodes  to  compute  the  Fv  and  Sv  values  (lines  3-22).  The  base 
case  applies  when  v  is  a  source  (i.e.,  leaf)  node.  In  this  case  we  can  satisfy  the  source 
node  if  and  only  if  we  use  flow  at  least  equal  to  bv.  Thus,  if  y  <  b  then  Fv(y)  =  0  and 
SJy)  =  0,  while  if  y  —  bv  then  Fjy)  =  1  and  Sv(y)  =  {n}  (lines  5-6).  Note  that  Fv  and 
and  Sv  do  not  depend  on  the  capacity  of  the  edge  (v,p(v));  this  constraint  will  be  taken 
into  account  when  we  compute  the  values  for  interior  nodes. 

For  an  interior  node  v,  we  recursively  compute  Fv(y)  based  on  the  values  Fu(z)  for  chil¬ 
dren  u  of  v.  If  y  total  flow  is  allowed  to  be  sent  to  v,  this  flow  must  come  from  the  children 
of  v  and  hence  we  would  like  search  the  space  of  all  possible  ways  of  dividing  the  y  flow 
for  v  among  the  children  of  v.  However,  naively  enumerating  these  takes  exponential  time. 
Instead,  we  use  another  level  of  dynamic  programming.  The  optimal  substructure  that  we 
exploit  is  that  the  optimal  way  to  divide  y  flow  among  /  children  must  be  some  combina¬ 
tion  of  optimally  dividing  between  z  flow  among  the  first  l  —  1  children  and  reserving  the 
remaining  y  —  z  flow  for  the  Zth  child,  for  some  z  G  [0..t/]. 

In  lines  13-21  we  iterate  through  the  children.  At  the  beginning  of  iteration  /,  we  have 
Fjr'  iy),  the  Fv  values  computed  by  considering  the  first  /  —  1  children,  and  the  corre¬ 
sponding  Sv  values;  we  compute  the  values  for  the  Zth  iteration  for  all  possible  y  (the  inner 
loop  on  lines  14-20).  For  a  given  value  of  y,  we  compute  Flv{y)  by  exhaustively  consid¬ 
ering  all  ways  to  divide  the  y  units  of  flow  into  z  units  of  flow  for  the  first  /  —  1  children 
and  the  y  —  z  units  of  flow  for  the  Zth  child  (the  loop  from  lines  16  -  20);  we  choose  the 
division  that  maximizes  the  number  of  satisfied  groups.  It  is  at  this  stage  that  we  check  for 
capacity  constraints  on  line  17.  the  amount  of  flow  that  can  be  obtained  from  the  child  u 
is  bounded  by  the  capacity  of  the  edge  w(u,  v )  and  Bu. 
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Algorithm  1  Tree-In 

Input:  Tr  =  ( V. ,  E,  t);  edge  capacities  w;  source  nodes  S  C  V  with  bs  for  all  s  E  S. 
Output:  Maximum  S'  C  S  that  satisfies  all  flow  constraints. 

1:  Set  Bv  to  be  the  sum  of  flow  requirements  in  the  subtree  rooted  at  v,  for  all  v  E  V 

2:  Order  V  by  the  depth-first  post  order 

3:  for  all  v  E  V  do 

4:  if  v  is  a  source  node  then 

5:  Fv(y)  E-  0  for  0  <  y  <  bv 

6:  Sv(y)  E-  0  for  0  <  y  <  bv 

7:  Fv(bv )  E-  1 

8:  Sv(bv)  E-  {n} 

9:  else  if  v  is  an  interior  node  then 

10:  F°(y)  E-  0  for  0  <  y  <  Bv 

11:  Sy(y)  E-  0  for  0  <  y  <  Bv 

12:  l  i —  1 

13:  for  all  child  u  of  v  do 

14:  for  y  E-  0  to  Bv  do 

15:  Flv{y)  E-  0 

16:  for  z  E-  0  to  y  do 

17:  A  •<— min (y  —  z,w(u,v),  Bu) 

18:  if  Flv{y)  <  Fl~\z)  +  FU{ A)  then 

19:  Flv(y)  E-  Fl~\z)  +  Ftt(A) 

20:  Slv(y)  E-  Sl~1(z)  U  SU(A) 

21:  l  i —  /  “hi 

22:  Fv(y)  E-  Fl~l{y )  for  0  <  y  <  Bv 

23:  Sv{y)  E-  Slv~1(y)  for  0  <  y  <  Bv 

24:  return  St(Bt ) 
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After  we  have  looped  through  all  children,  we  take  Fv  and  Sv  to  be  the  values  of  we 
computed  in  the  final  iteration  (lines  22  and  23),  as  these  are  the  values  after  considering 
all  of  v’s  children.  Note  that  it  is  important  that  we  compute  Fv(y)  and  Sv(y)  for  all 
possible  values  of  y.  This  is  because  we  will  need  to  use  them  when  considering  the 
parent  of  v,  which  must  consider  all  possible  ways  to  divide  its  flow  between  its  children 
(i.e.,  v  and  v’s  siblings). 

The  running  time  of  Algorithm  1  is  pseudo-polynomial.  It  is  clear  that  we  can  compute 
lines  1  and  2  in  0 ( n  +  m)  time.  Observe  that  Bt  =  max,,  Bv.  Within  the  main  body  of  the 
algorithm,  the  innermost  loop  (lines  16-20)  takes  at  most  0(Bt )  time  and  it  is  repeated  at 
most  0(Bt )  time  in  the  loop  from  lines  14  -  20.  As  this  loop  in  turn  is  repeated  for  each 
child  of  interior  node,  the  innermost  loop  is  repeated  at  most  0(n)  times  in  total,  for  a  time 
of  O(nBf  ').  The  rest  of  the  outermost  loop  body  in  lines  4-23  take  at  most  0(Bt )  time,  so 
the  total  running  time  is  0(nB‘f  +  m),  which  is  pseudo-polynomial  in  the  size  input  due 
to  its  polynomial  dependence  on  the  magnitude  of  Bt. 


2.3  Unsplittable  Flow  Allocation 

In  this  section  we  consider  the  case  where  the  flows  cannot  be  divided  over  multiple  paths 
as  they  could  be  in  the  previous  problem  formulations.  Instead,  the  flow  from  each  source 
node  must  be  routed  along  a  single  path  to  the  sink  node.  This  assumption  may  be  the 
case  for  some  communication  networks  and  is  also  applicable  when  the  flows  represent 
the  movement  of  physical  agents  (such  as  a  robot)  that  cannot  be  divided.  We  consider 
consider  general  directed  graph  inputs  and  seek  to  maximize  the  number  of  source  groups 
that  can  fully  transmit  their  required  flows  to  the  sink. 

Problem  3.  (MaxSG-UF)  Given  a  directed  graph  G  =  (V.  E )  with  integral  edge  capacities 
w{e)  >  0  for  all  e  e  E,  source  groups  6,  integral  source  requirements  b,  and  sink  nodes 
T,  find  AC©,  sink  assignment  x,  and  satisfying  unsplittable  flow  /  for  X  such  that  \X\ 
is  maximized. 

The  decision  problem  associated  with  this  optimization  problem,  MaxSG-UF-Dec,  is  to 
decide  whether  a  MaxSG-UF  instance  has  a  solution  with  \X\  >  k. 

Theorem  2.  The  MaxSG-UF-Dec  decision  problem  is  strongly  NP -complete. 

Proof.  Given  a  flow  /  from  the  source  nodes  to  sinks,  it  is  straightforward  to  check  in 
polynomial  time  whether  they  meet  the  flow  requirements  for  at  least  k  groups,  so  this  is 
clearly  in  NP. 

We  prove  strong  NP-hardness  by  reduction  from  3 -Partition,  which  is  known  to  be  strongly 
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Figure  2.4:  MaxSG-UF-Dec  is  strongly  NP-complete:  Reduction  from  3-Partition. 


NP-complete  [21].  The  3-Partition  decision  problem  is  as  follows:  Given  a  set  A  = 
{ai, . . . ,  a3m}  of  3m  positive  integers  where  a  =  mC,  and  C/4  <  a  <  C/2  for  all 
a  G  A,  decide  whether  A  be  partitioned  into  m  subsets  Ai, . . . ,  Arn  such  that  Y/a&A  cl  —  C 
for  all  Ai. 

Given  a  3-Partition  instance,  we  construct  a  MaxSG-UF-Dec  instance,  as  illustrated  in 
Figure  2.4.  We  create  a  sink  node  t,  then  add  m  intermediate  nodes  v\, . . . ,  vrn  and  connect 
each  one  to  t  with  edge  (vi,t)  with  capacity  w(vi,  t )  =  C.  We  then  create  3m  source  nodes 
Si, . . . ,  s3m  and  for  all  source  nodes  st  and  intermediate  nodes  v3,  we  add  an  edge  (si;  v3 ) 
with  capacity  C.  We  set  the  source  requirements  bSi  =  x,  for  1  <  i  <  3m,  and  ask  whether 
it  is  possible  to  satisfy  all  k  =  3m  source  groups. 

Suppose  that  the  3-Partition  decision  is  YES.  Then  for  each  Ai  =  {«/,,  , ,  a/,i2 ,  }  we  have 

that  akn  +  (ik,2  +  aki3  =  C  and  therefore  we  can  route  the  flow  from  skil ,  Ski2 ,  and  Ski3  to  t 
via  node  vt.  Because  the  Ai  cover  A,  it  is  possible  to  satisfy  the  source  requirements  for  all 
source  nodes.  Because  the  At  are  pairwise  disjoint,  each  a3  e  /I  is  in  exactly  one  /l,  and 
hence  it  is  possible  for  s3  e  S  to  send  its  flow  only  to  vr .  Thus  the  flow  for  each  source 
node  can  be  routed  on  a  single  path  to  t,  and  so  the  MaxSG-UF-Dec  decision  is  YES. 

Conversely  assume  that  the  MaxSG-UF-Dec  decision  is  YES.  From  the  construction  of 
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capacities  and  source  requirements,  each  intermediate  node  must  receive  flow  from  ex¬ 
actly  three  children  {skn,Ski2,Ski3},  and  their  combined  flow  must  equal  C.  Further¬ 
more,  no  source  node  sends  flow  to  more  than  one  intermediate  node.  Hence  the  sets 
Ai  =  {a^i,  Ufci2,  a/,-,3 }  satisfy  the  3-Partition  requirements,  and  so  the  3-Partition  decision 
is  YES. 

Thus  the  3-Partition  decision  is  YES  if  and  only  the  constructed  MaxSG-UF-Dec  decision 
is  YES,  and  so  MaxSG-UF-Dec  is  strongly  NP-complete.  □ 

By  Theorem  2  there  is  no  known  pseudopolynomial-time  algorithm  for  solving  MaxSG- 
UF  optimally.  We  next  show  that  this  problem  is  hard  to  even  approximate. 

Theorem  3.  It  is  NP-lmrd  to  approximate  MaxSG-UF  within  a  factor  of  Kl  '  for  any 
e  >  0  where  m  is  the  number  of  edges  of  the  graph  in  the  MaxSG-UF  problem. 


Proof  The  proof  is  by  reduction  from  the  NP-hard  problem  2DIRPATH  [26].  An  in¬ 
stance  of  2DIRPATHis  a  directed  graph  FI  =  (VH,  Eh)  and  distinct  nodes  x1,  x2,  y1,  y2  G 
Vh  ■  The  problem  is  to  decide  whether  there  exist  directed  paths  from  x1  to  if  and  x2  to  y 2 
such  that  the  paths  share  no  common  edges  (i.e.,  they  are  edge-disjoint). 


The  reduction  is  a  modified  version  of  the  one  used  in  [26].  Given  any  e  >  0,  we  construct 
a  directed  graph  G  from  H.  Figure  2.5  provides  an  illustration  of  the  construction. 

Let  N  =  |Vf/|^/el-  We  first  create  N  source  nodes  s*  for  i  G  [l..iV],  with  source  require¬ 
ments  hSi  =  i.  We  place  each  source  node  in  its  own  group  so  that  6  =  {{s}  :  s  G  S}. 
We  then  create  N  sink  nodes  t,  for  i  G  [l..iV].  Place  source  nodes  from  right  to  left  on 
horizontal  axis,  s,  at  position  n  —  i+1  and  sink  nodes  from  bottom  to  top  on  vertical  axis, 
ti  at  position  i,  where  1  <  i  <  N .  As  in  Figure  2.5,  we  connect  s,  to  t,  by  draw  a  line 
from  Si,  then  going  up  until  a  node  dlA  at  height  i,  then  going  left  until  connecting  ti. 


by  adding  a  capacity  of  i  to  each  edge.  The  nodes  corresponding  to  xl,x 
are  denoted  x\6,  x2j,  yf,  y?- 


For  each  pair  1  <  j  <  i  <  N  we  create  a  modified  version  of  H  that  we  denote  //ZJ 

2,y1,y2  in  H 

respectively,  in  HtJ.  Suppose  we  wish  to  send  j'  <  j  units 
of  unsplittable  flow  from  xf  to  y]-  and  i  units  of  unsplittable  flow  from  x2VJ  to  y2-.  This 
is  possible  if  and  only  if  there  exist  edge-disjoint  paths  in  H  from  x1  to  if  and  x1  to  y2, 
because  each  edge  e  in  IItj  can  only  carry  flow  from  one  of  xf  or  xf  due  to  the  capacity 
constraint  w(e)  =  1,  and  the  flows  from  each  of  xf  and  xf  cannot  split  and  must  therefore 
follow  single  paths. 


Returning  now  to  our  construction  of  G,  we  place  the  graph  IILJ  at  the  intersection  of  the 
path  from  Sj  to  ti  and  the  path  from  $:J  to  tj  where  1  <  j  <  i  <  n.  We  connect  the  path 
from  Si  to  x2j  and  y2-  to  the  path  to  dj.  We  also  connect  the  path  from  dj  to  x}j  and  y\-  to 
the  path  to  tr  Formally  this  is  achieved  in  the  following  way.  For  i  G  [1..  Ar]  we  add  edges 
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Figure  2.5:  MaxSG-UF  is  hard  to  approximate:  Reduction  from  2DIRPATH. 
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(si,x2f)  and  (y^.^,  dj)  with  capacities  i.  For  j  e  [1..7V]  we  add  edges  (d^,  x^-+1^)  and 
( VNjitj )  with  capacities  j.  For  all  2  <  j  <  i  <  N  —  1,  we  add  edges  (y^-ip x%  with 
capacity  i  and  xjj)  with  capacity  j. 

CLAIM  1.  If  there  exist  edge-disjoint  paths  from  x 1  to  y1  and  x2  to  y2  in  H,  then  all 
source  groups  in  G  can  be  satisfied. 

Proof  of  Claim  1.  Suppose  there  exist  such  edge-disjoint  paths  in  H.  By  our  earlier  ob¬ 
servation,  it  follows  that  it  is  possible  to  route  j  units  of  flow  from  x\3  to  y]-  and  i  units  of 
flow  from  x23  to  y2-  in  7/y  for  all  1  <  j  <  i  <  N.  By  construction  of  the  other  edges  of 
G,  for  each  s*  G  S'  it  is  therefore  possible  to  send  bSi  =  i  units  of  flow  on  the  path  from  s; 
to  di  by  taking  edges  “up”  in  Figure  2.5,  then  from  dt  to  U  by  going  “left.”  Since  each  s; 
forms  its  own  group  in  <3,  it  follows  that  all  source  groups  can  be  satisfied.  □ 

CLAIM  2.  If  there  do  not  exist  edge-disjoint  paths  from  x1  to  if  and  x2  to  y2  in  II.  then 
at  most  one  source  group  in  G  can  be  satisfied. 

Proof  of  Claim  2.  We  prove  this  claim  by  contradiction.  Suppose  that  two  source  groups 
{si}  and  {sj}  can  be  satisfied,  with  3  <  i.  Observe  that  by  the  construction  of  capacities, 
the  flow  from  sl  must  pass  through  di,  and  the  flow  from  Sj  must  pass  through  dj.  Because 
j  <  i,  it  follows  that  the  flow  from  Sj  must  “cross”  the  flow  from  s,  at  some  II,y  with 
j  <  j'  <  i.  That  is,  there  exists  j'  with  j  <  j'  <  i  such  that  i  units  of  flow  from  st  enters 
x\y  and  exits  yf,  and  j  units  of  flow  from  s3  enters  x}y  and  exits  yf,.  By  assumption  H 
does  not  contain  edge-disjoint  paths  from  x1  to  y1  and  x2  to  y2  and  so  by  our  previous 
observation,  it  is  not  possible  to  route  the  desired  flows  through  H%y.  But  this  means  that 
it  is  not  possible  to  satisfy  both  {s*}  and  {s^},  which  is  a  contradiction,  thus  completing 
our  proof  of  the  claim.  □ 

By  the  above  two  claims,  YES  instances  of  2DIRPATHare  mapped  to  instances  of  MaxSG- 
UF  where  all  N  source  groups  can  be  satisfied,  while  NO  instances  are  mapped  to  instances 
of  MaxSG-UF  where  at  most  one  source  group  can  be  satisfied.  Suppose  we  had  an  ap¬ 
proximation  algorithm  with  a  ratio  K 1-e.  Then  given  an  instance  of  2DIRPATH,  we 
could  construct  an  instance  of  MaxSG-UF  with  N  source  groups  as  described  above  in 
polynomial  time.  We  know  that  it  must  be  possible  to  either  satisfy  all  the  source  groups, 
or  satisfy  at  most  one  of  them.  We  then  run  the  approximation  algorithm. 

If  all  N  source  groups  can  be  satisfied,  the  approximation  ratio  guarantees  that  we  would 
be  able  to  find  a  solution  with  at  least  N/N1~e  =  Ne  groups  satisfied.  Because  N  = 
|V|I'1/£1,  we  are  thus  guaranteed  to  find  a  solution  with  at  least  \V\  groups  satisfied.  By 
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assumption  \  V\  >  4  (the  starting  and  ending  nodes  in  the  2DIRPATHmust  all  be  distinct) 
so  we  can  satisfy  at  least  4  groups. 

If,  on  the  other  hand,  at  most  one  source  group  can  be  satisfied,  the  approximation  algo¬ 
rithm  will  also  only  be  able  to  find  a  solution  that  satisfies  at  most  one  group.  Therefore, 
by  seeing  if  the  approximation  algorithm  can  satisfy  more  than  one  group  we  will  be  able 
to  decide  2DIRPATH.  But  2DIRPATHis  NP-hard,  and  so  approximating  MaxSG-UF 
within  K 1-e  is  also  NP-hard.  This  completes  the  proof  of  Theorem  3.  □ 

Theorem  3  essentially  rules  out  any  non-trivial  approximation  algorithm  with  a  provable 
performance  guarantee.  The  best  that  we  can  hope  to  do,  unless  P  =  NP,  is  to  consider 
each  group  in  isolation  and  try  to  find  a  satisfying  flow,  even  when  each  group  contains 
only  a  single  source  node.  In  that  case,  finding  a  satisfying  flow  reduces  to  finding  a 
directed  path  from  the  source  node  s  to  a  sink,  using  only  edges  with  capacity  w{e)  >  bs. 
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Chapter  3 

Network  Augmentation  and  Flow 
Allocation 


In  this  chapter  we  consider  environments  that  the  agents  can  modify  by  adding  nodes 
and  edges  to  improve  connectivity  and  increase  capacity,  a  process  we  term  network  aug¬ 
mentation.  This  is  primarily  motivated  by  the  challenge  of  maintaining  effective  wire¬ 
less  communication  networks  for  multiagent  teams.  Agents  must  often  operate  in  noisy, 
obstacle-filled  environments  equipped  with  communication  hardware  that  is  size-  and 
power-limited,  leading  to  poor  network  performance.  One  way  to  remedy  this  is  to  sup¬ 
plement  the  agent-based  network  by  deploying  additional  nodes  to  act  as  relays. 

Agents  can  also  affect  physical  environments,  such  as  the  road  network  in  an  urban  dis¬ 
aster  response  domain.  For  example,  police  officers  may  be  deployed  to  direct  vehicular 
traffic,  thereby  reducing  congestion  and  effectively  increasing  the  road  network’s  capacity 
to  transport  resources.  In  another  example,  robots  may  be  deployed  to  restore  edges  in  the 
transportation  network  by  clearing  away  debris  from  obstructed  roads. 

The  key  problem  in  network  augmentation  is  to  determine  where  to  best  deploy  the  addi¬ 
tional  nodes.  This  partly  depends  on  factors  intrinsic  to  the  environment:  which  agents  will 
be  in  range  if  a  relay  is  placed  in  a  specific  spot,  for  example,  or  what  roads  are  currently 
obstructed  but  can  be  cleared.  It  also  depends  on  the  flows  through  the  environment:  low 
capacity  only  becomes  a  bottleneck  if  it  is  being  used.  But  the  choice  of  flows  also  depends 
on  the  environment,  and  hence  on  the  choice  of  network  augmentation.  This  highlights  the 
interdependence  of  the  network  augmentation  and  flow  allocation  problems.  Because  of 
this  interdependence,  in  this  chapter  we  simultaneously  optimize  both  the  placement  of 
supplemental  nodes  and  the  allocation  of  flows. 
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As  in  the  previous  chapter,  we  represent  the  environment  as  a  directed  graph  with  ca¬ 
pacities  on  edges.  We  consider  the  initial  network  to  be  the  graph  of  the  environment 
without  any  supplemental  nodes.  We  explicitly  represent  the  ways  in  which  the  agents  can 
augment  the  network  by  the  potential  network ,  which  adds  to  the  initial  networks  a  set 
of  nodes  representing  the  locations  where  supplemental  nodes  can  be  deployed  to,  along 
with  additional  edges  indicating  the  connectivity  that  would  result  if  supplemental  nodes 
were  deployed  to  those  locations.  The  potential  locations  are  chosen  a  priori  possibly  by 
domain  specialists  or  domain- specific  algorithms  (such  as  identifying  feasible  locations 
for  airborne  relays,  or  the  locations  obstructed  roads),  or  as  finite  approximations  (in  the 
case  of  relays  positioned  in  a  continuous  space). 

In  this  chapter  we  formalize  the  problem  of  network  augmentation  and  flow  allocation, 
analyzing  its  complexity  and  providing  algorithms  for  its  solution.  A  summary  of  the 
notation  used  in  this  chapter  is  provided  in  Table  3.1.  Portions  of  the  work  in  this  chapter 
was  originally  published  in  Okamoto  and  Sycara  [48]. 


3.1  Network  Augmentation 

We  know  that  the  problem  of  network  augmentation  and  flow  allocation  is  NP-hard,  be¬ 
cause  the  flow  allocation  problem  formulated  in  the  previous  chapter  is  a  special  case 
where  the  initial  and  potential  networks  are  identical.  However,  in  this  section  we  con¬ 
sider  a  form  of  “pure”  network  augmentation  that  removes  as  many  of  the  features  of  flow 
allocation  as  possible  in  order  to  focus  on  the  issues  inherent  to  the  network  augmenta¬ 
tion  side  of  the  problem.  We  show  that  even  this  simplified  pure  network  augmentation 
problem  is  NP-hard. 

In  pure  network  augmentation,  we  are  given  an  initial  network  topology,  a  finite  set  of 
potential  locations  where  supplemental  nodes  can  be  deployed,  and  groups  of  source  nodes 
representing  agents  that  need  to  communicate.  The  simplest  case  of  network  augmentation 
arises  when  all  edges  have  unlimited  (i.e.,  positive  infinity)  capacity  and  are  symmetric. 
The  problem  is  to  deploy  supplemental  nodes  to  the  potential  locations  in  order  to  satisfy 
group  requirements.  In  this  case,  group  satisfaction  reduces  to  connectivity,  so  a  group  of 
source  nodes  is  satisfied  if  all  the  nodes  in  the  group  are  connected  to  each  other  once  the 
supplemental  nodes  have  been  deployed. 

Let  the  initial  network  topology  be  represented  as  an  undirected  simple  graph  G  =  (V,  E), 
where  V  is  the  set  of  nodes,  and  there  exists  an  edge  (u,  v)  (=  E  if  and  only  if  u  and  v 
can  directly  communicate.  Denote  the  potential  network  by  the  undirected  simple  graph 
G'  =  ( V',E ')  =  (V  U  P,  E  U  EP).  The  set  of  possible  locations  where  supplemental 
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Symbol 

Description 

A 

-^uv 

binary  indicator  variable  that  potential  edge  (u,  v )  G  EP  is 
in  the  actual  network  so  (w,  u)  G  ED 

K 

source  requirement  of  s  G  Si 

D  CP 

deployment  of  supplemental  nodes 

Dp 

binary  indicator  variable  that  p  G  D 

E 

set  of  edges 

EP 

set  of  potential  edges,  i.e.,  edges  with  at  least  one  endpoint 
in  P 

G=(V,E ) 

initial  network  with  nodes  V  and  edges  E 

G'  =  (V  U  P,  E  U  EP) 

potential  network  with  initial  nodes  V,  potential  nodes  P, 
initial  edges  E,  and  potential  edges  EP 

Gp  =  (Vd,  Ep) 

actual  network  for  a  deployment  D:  the  subgraph  of  6"  in¬ 
duced  by  C  U  D 

h 

maximum  number  of  supplemental  nodes  allowed  in 
MaxSG-NA 

K 

number  of  source  groups 

m 

number  of  edges 

n 

number  of  nodes 

S  C  V 

set  of  source  nodes 

Si  g  6 

ith  group  of  source  nodes 

&  =  {SU...,SK} 

set  of  groups  of  agents  that  partitions  the  agents 

T c  VCP\S 

set  of  sink  nodes 

V 

set  of  initial  nodes 

Xi  g  T 

sink  node  assigned  to  group  i,  if  any 

Xu 

binary  indicator  variable  that  sink  t  G  T  is  assigned  to  group 

Si  G  6 

Table  3.1:  Summary  of  notation  for  Chapter  3 
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nodes  be  deployed  is  denoted  by  P,  with  V  P  =  0,  and  the  set  of  potential  communication 
links  is  denoted  by  Ep,  with  Ep  E  —  0.  For  each  edge  (u,v)  E  Ep  ,  either  u  E  P  or 
v  E  P  or  both.  If  u  E  P  and  v  cj  P,  then  (u,  v )  E  Ep  represents  that  a  supplemental 
node  positioned  at  u  can  communicate  with  node  v;  similarly,  if  u  (f  P  and  v  E  P,  then 
(u,  v )  E  EP  means  that  a  supplemental  node  positioned  at  v  can  communicate  with  node 
u;  and  finally,  if  both  u.  v  E  P,  then  if  supplemental  nodes  were  deployed  to  both  u  and  v, 
they  would  be  able  to  communicate. 

We  represent  a  deployment  of  supplemental  nodes  by  D  C  P,  the  set  of  potential  locations 
where  we  have  deployed  supplemental  nodes.  We  denote  the  actual  network  instantiated 
by  D  by  the  undirected  simple  graph  G  p  =  (  I'd  ,  E n )  that  is  the  subgraph  of  6"  induced 
byVUD. 

Let  S  be  the  set  of  source  nodes,  and  &  =  {£1,  S 2, . . . ,  Sk}  be  the  set  of  source  groups, 
where  S',  C  S  for  i  =  1, . . . ,  K .  We  would  like  all  of  the  nodes  within  a  group  S,  E  S  to 
be  connected  to  each  other  in  Gp >.  We  refer  to  this  property  as  group  connectivity,  and  say 
that  the  group  5}  is  connected  (or  is  a  connected  group). 

There  are  two  related  network  augmentation  optimization  problems,  depending  on  the 
objective  function.  The  Minimum  Deployment  Connected  Groups  problem  seeks  to  min¬ 
imize  the  number  of  supplemental  nodes  required  to  satisfy  all  groups.  The  Maximum 
Connected  Groups  problem  seeks  to  maximize  the  number  of  satisfied  groups  given  a 
fixed  number  of  supplemental  nodes. 

Problem  4.  Minimum  Deployment  Connected  Groups  (MinDepCG).  Given  an  initial 

network  G  =  (V.  E),  potential  network  G'  —  (V  U  P,  E  U  Ep),  and  source  groups  &  = 
(Si, . . .  ,Sk},  find  a  deployment  D  with  minimum  \D\  such  that  all  of  the  groups  are 
connected  in  Gp. 

Problem  5.  Maximum  Connected  Groups  (MaxCG).  Given  an  initial  network  G  = 
(V,  E),  potential  network  G'  =  (V  U  P,  E  U  EP),  source  groups  6  =  {^i, . . . ,  and 
integer  constant  h  >  1,  find  a  deployment  D  with  \D\  <  h  such  that  the  maximum  number 
of  groups  are  connected  in  Gp- 

The  decision  problem  related  to  MinDepCG  and  MaxCG  is  the  Connected  Groups  prob¬ 
lem: 

Problem  6.  Connected  Groups  (CG).  Given  an  initial  network  G  =  {V,E),  potential 
network  G'  =  (V  U  P,  E  G  EP ),  agent  groups  &  =  {^i, . . . ,  5a-},  and  integer  constants 
h  >  1  and  k  >  1,  is  there  a  deployment  D  with  \D\  <  h  such  that  at  least  k  groups  are 
connected  in  Gd? 

We  next  show  that  the  Connected  Groups  problem  is  NP-complete,  even  for  a  single  group. 
Theorem  4.  CG  is  NP-complete. 
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Proof.  To  show  this  we  must  show  that  CG  is  in  NP  and  is  NP-hard.  It  is  clearly  in  NP, 
as  we  can  easily  verify  in  polynomial  time  whether  a  set  of  possible  locations  D  C  P  is  a 
solution  or  not. 

Proof  of  NP-hardness  is  done  by  reduction  from  the  graph  Steiner  tree  (ST)  decision  prob¬ 
lem  with  unit-weight  edges,  which  is  known  to  be  NP-complete  [20].  The  input  to  ST  is 
an  undirected  graph  GSt  =  (Vst,  Es t),  a  set  of  terminals  XST  C  VSt,  and  a  maximum 
cost  ksr-  The  problem  is  to  decide  whether  there  exists  a  tree  Tst  =  (WST,  FST)  which 
is  a  subgraph  of  GSt  with  XST  C  WSt  and  at  most  kST  edges.  Such  a  tree  Tst  is  called 
a  Steiner  tree. 

From  an  ST  input  instance  Gst ,  XST,  ksr  for  ST,  we  construct  the  following  CG  input 

G,G',&,h: 

•  G  =  (V,  E )  where  V  =  Xst,  and  E  —  {(u,  v )  G  ESt\u  G  XSt  and  v  G  XSt}- 

•  G'  =  (V  UP,E\J  EP ),  where  P  =  Vst  \  XSt  and  EP  =  {(u,v)  G  ESt\u 
Xst  or  v  Xst}- 

•  &  =  {/Si},  where  .S')  =  XSr- 

•  h  =  ksT  +  1  — 

•  k  —  1. 

We  now  show  that  Gst  has  a  Steiner  tree  with  at  most  ksr  edges  if  and  only  if  the  con¬ 
structed  CG  instance  has  a  solution. 

Suppose  first  that  Gst  has  a  Steiner  tree  Tst  =  (Wst,  Tst )  with  at  most  ksT  edges. 
We  will  show  that  there  is  a  deployment  that  satisfies  the  solution  conditions  for  the  CG 
instance.  Set  D  =  WSt  \  XST ■  Then  clearly 

D  =  Wst  \  XSt 

Q  Vst  \  Xst  ( Xst  Q  Vst) 

=  P  (by  construction) 

Note  that  .S',  =  XSt  is  connected  in  the  subgraph  of  G'  induced  by  C  U  D,  since  XST  is 
connected  in  Tst. 

We  also  get  that 


|  Wst  |  =  \Fsr\  +  1 

<  ksT  +  1 


(TSt  is  a  tree) 
(by  assumption) 
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so  that 


\D\  =  | Wst  \  XST\ 

=  |  Wst  |  _  |Xst|  {Xst  C  Wst) 

<  ksT  +  1  —  Xst 

=  h  (by  construction) 

Thus  D  is  a  solution  to  CG. 

Assume  now  instead  that  there  is  a  solution  D  to  a  CG  instance  as  constructed  above.  We 
will  show  that  there  must  then  be  a  Steiner  tree  with  at  most  ksr  edges  for  the  Steiner 
instance.  Let  Gd  =  (Vd,  Ed)  denote  the  subgraph  of  G'  induced  by  I).  Then 

|Vb|  =  | Vd  (T  A"st|  +  |Vd  \  Xst 
—  I^stI  +  | Vd  \  Xst\ 

=  \Xsr\  +  \VDnP\ 

=  \XST\  +  \D\ 

<  \XSt\  +  h 

=  |  Xst  |  +  ksT  +  1  —  |  Xst 
=  ksT  +  1- 

Because  D  is  a  solution  to  CG,  the  nodes  in  =  XSt  are  connected  in  GD.  Let  TD  — 
(Vb,  ETd)  denote  a  spanning  tree  of  GD.  Because  the  nodes  in  XST  are  connected  in  GD, 
it  follows  that  they  are  also  connected  in  Td,  and  so  Td  is  a  Steiner  tree.  Now  we  get  that 

| ETd  =  |Vb|  -  1  (Td  is  a  tree) 

<  kST  +  1  —  1  (by  above) 

=  ksT 

Thus  To  is  a  Steiner  tree  with  at  most  ksT  edges. 

Hence  GsT  has  a  Steiner  tree  with  at  most  ksT  edges  if  and  only  if  the  constructed  CG 
instance  has  a  solution.  Therefore  CG  is  NP-complete.  □ 


(■ XSt  G  Vd) 

(P  =  Vst\XSt) 

(D  is  a  solution  to  CG) 
(by  construction) 


Because  other  variations  of  the  Network  Augmentation  problem  with  the  maximum  group 
or  minimum  deployment  objective  functions  are  generalizations  of  this  simple  case,  it 
follows  that  they  too  are  NP-complete. 
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3.2  Network  Augmentation  and  Flow  Allocation 


We  now  consider  the  full  problem  of  network  augmentation  and  flow  allocation.  This  is 
the  problem  of  choosing  a  deployment  and  selecting  a  sink  and  satisfying  flow  for  each 
group.  It  is  a  generalization  of  both  the  pure  network  augmentation  and  flow  allocation 
problems. 

As  in  pure  network  augmentation,  we  are  given  an  initial  network  G  =  ( V ,  E )  and  poten¬ 
tial  network  G'  =  [V ,  E')  =  (V  U  P.  E  U  EP),  but  these  are  now  capacitated,  directed 
graphs,  with  the  capacity  of  each  edge  e  £  E'  denoted  by  we,  as  in  the  flow  allocation 
problem.  Capacities  may  be  either  positive,  real  values  (representing  limited  capacity), 
or  may  be  positive  infinity  (representing  unlimited  capacity).  A  deployment  D  C  Pis 
the  set  of  potential  location  where  supplemental  nodes  are  deployed.  The  actual  network 
for  a  deployment  D  is  the  graph  GD  =  (VD,  ED)  that  is  the  subgraph  of  6"  induced  by 
VD  =  V  U  D. 

As  in  pure  flow  allocation,  source  nodes  S  C  V  are  partitioned  into  groups  6  =  {.S') , . . . ,  5V}, 
each  of  which  must  send  flow  to  a  single  sink.  The  flow  requirement  b\  is  the  amount  of 
flow  that  must  be  sent  by  source  node  s  £  S).  The  set  of  sinks  is  denoted  by  T  cfUP. 

If  v  £  T  (T  V,  it  means  that  the  node  v  can  act  as  a  sink,  while  if  p  £  T  fl  P,  it  means  that 
a  supplemental  node  deployed  to  p  can  act  as  a  sink.  The  set  of  groups  for  which  flow  has 
been  allocated  is  A  C  ©,  and  the  sink  assigned  to  group  ,5)  £  A''  is  xt  £  T.  The  amount 
of  flow  on  edge  (u,  v )  for  group  S)  is  denoted 

A  feasible  solution  is  a  deployment  D,  sink  assignments  x,  and  flow  /  such  that  /  is  a 
satisfying  flow  for  X  in  the  actual  network  Go-  This  means  that  none  of  the  groups  can  be 
assigned  to  sinks  that  are  not  in  Go,  and  no  flow  can  be  transmitted  on  any  potential  edge 
that  is  not  in  GP-  Formally,  these  constraints  are 

Xi.eTn(VUD)  VSiEX  (3.1) 

fl  =  0  V(u,v)  ^  ED,\/Si  £  X.  (3.2) 

These  constraints  must  be  met  in  addition  to  the  usual  constraints  in  Equations  (2.1)  -  (2.3) 
for  satisfying  flows. 

As  in  pure  network  augmentation,  there  are  two  optimization  problems  depending  on  the 
choice  of  objective  function.  Maximum  Satisfied  Groups  with  Network  Augmentation 
(MaxSG-NA)  is  the  problem  of  satisfying  the  greatest  number  of  groups  with  a  fixed  num¬ 
ber  of  supplemental  nodes.  The  Minimum  Deployment  (MinDep)  problem  is  to  satisfy  all 
of  the  groups  using  the  fewest  number  of  supplemental  nodes  possible. 

Problem  7.  Maximum  Satisfied  Groups  with  Network  Augmentation  (MaxSG-NA). 
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Given  initial  network  G  =  (V,  E),  potential  network  G'  =  (V.  E')  —  (V  U  P.E  U  EP), 
capacities  w,  source  groups  &  =  {Si, . . . ,  Sk},  source  requirements  b,  sinks  T  C  V  U  P, 
and  number  of  supplemental  nodes  h  >  0,  find  a  deployment  D,  subset  of  groups  X  C  (5, 
sink  assignment  x,  and  satisfying  flow  /  for  X  in  Go  such  that  \X\  is  maximized 
Problem  8.  Minimum  Deployment  (MinDep).  Given  initial  network  G  =  (  V.  E),  po¬ 
tential  network  G'  =  (V',E')  =  (V  UP,fU  EP),  capacities  w,  source  groups  &  = 
{Si, . . . ,  SK},  source  requirements  b,  and  sinks  T  C  f  UP,  find  a  deployment  D,  sink 
assignment  x,  and  flow  /  for  X  in  Gp  and  satisfying  flow  /  for  &  in  Gp  such  that  \D\  is 
minimized. 

Theorem  5.  MaxSG-NA  and  MinDep  are  NP-hard. 

Proof.  This  follows  from  MaxSG-NA  and  MinDep  generalizing  MaxSG  and  CG,  both  of 
which  are  NP-hard.  □ 


3.3  Solving  MaxSG-NA  and  MinDep 

In  this  section  we  present  algorithms  for  solving  MaxSG-NA  and  MinDep.  To  solve  the 
problems  optimally  we  use  a  MILP  formulation. 


3.3.1  Optimal  Algorithms 

We  optimally  solve  MaxSG-NA  and  MinDep  by  formulating  the  problems  as  MILPs 
which  can  then  be  solved  using  standard  MILP  solvers.  This  approach  simultaneously 
optimizes  both  the  network  augmentation  and  flow  allocation  parts  of  the  problems.  The 
central  issue  is  to  convert  the  constraints  in  Equations  (3.1)  and  (3.2)  into  linear  constraints. 
To  represent  the  deployment  D,  we  use  binary  indicator  variables  Dp  for  p  e  P,  where 
Dp  =  1  if  p  G  P  and  Dp  =  0  otherwise.  As  in  MILP  1  for  flow  allocation,  we  represent 
sink  assignments  using  binary  indicator  variables  Xit  for  all  S,  e  ©,  t  e  T,  where  Xit  =  1 
if  t  is  the  sink  for  group  Si  and  Xlt  =  0  otherwise. 

In  representing  the  constraint  in  Equation  (3.1),  note  that  the  definition  of  the  Xit  variables 
already  constrains  x%  to  be  in  T,  and  furthermore  if  t  G  V  then  there  is  no  need  for  an 
additional  constraint.  For  t  &  P,  we  would  like  to  express  the  constraint  that  Xlt  can  take 
the  value  1  only  if  D,  =  1.  This  can  be  expressed  by  the  constraint  Xlt  <  Dt  for  all 
t  G  T  fl  P.  We  must  also  ensure  that  at  most  one  sink  is  assigned  to  each  group;  this  is 
expressed  as  ]G/gT Xlt  <  1  for  all  t  e  [1  ..K],  just  as  in  MILP  1.  There  is  no  need  to 
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explicitly  represent  X ,  because  that  the  composition  of  X  is  already  determined  by  the  Xlt 
variables:  X  contains  group  St  if  and  only  if  Xit  =  1  for  some  t  G  T  fl  Vp. 

The  constraint  in  Equation  (3.2)  requires  that  flow  only  be  sent  on  edges  in  the  actual 
network  Gp •  There  is  no  need  to  modify  the  flow  capacity  constraint  for  edges  in  the 
initial  network,  as  they  are  always  included  in  the  actual  network.  For  each  potential  edge 
e  =  (u,v)  G  EP,  we  add  a  binary  indicator  variable  Auv  (also  denoted  Ae)  that  will  be  1 
if  ( u ,  v )  is  present  in  G p ,  and  0  otherwise.  We  then  constrain  the  total  amount  of  flow  on 
(u,  v )  to  be  no  more  than  wuvAuv,  which  is  equal  to  wuv  if  (u,  v)  G  ED  and  0  otherwise, 
as  desired.  This  expression  is  linear  as  wuv  is  an  input  parameter. 

We  can  distinguish  two  kinds  of  potential  edges:  those  that  have  a  potential  location  at 
only  one  end  point,  and  those  that  have  potential  locations  at  both  end  points.  An  edge 
with  a  potential  location  at  only  one  end  point  has  the  form  e  =  (u.  p )  G  Ep  where  u  G  V 
and  p  G  P,  or  e  =  (p,  v )  G  Ep  where  v  G  V  and  p  G  P.  In  either  of  these  cases,  the 
presence  of  the  e  in  GD  is  indicated  by  the  presence  of  p  in  GD,  and  so  we  constrain  Ae  to 
equal  Dp. 

An  edge  with  potential  locations  at  both  end  points  is  slightly  trickier  to  represent.  In  this 
case  the  edge  has  the  form  (p.  q )  G  Ep  where  both  p  and  q  are  in  P.  Thus  the  expression 
for  the  presence  or  absence  of  e  in  the  actual  network  is  DpDq,  which  is  1  if  they  are  both 
present  and  0  otherwise.  Unfortunately,  DpDq  is  not  linear  but  quadratic,  and  so  it  cannot 
be  used  in  a  MILP  constraint.  Instead  we  introduce  a  binary  indicator  variable  Apq  for 
(p,  q)  G  EP  with  both  p.  q  G  P.  We  constrain  Apq  <  Dp  and  Apq  <  Dq;  this  ensures  that 
Apq  must  be  0  unless  both  p,q  G  D,  in  which  case  it  can  be  1 .  We  further  add  a  constraint 
Apq  >  Dp  +  Dq  —  1;  this  ensures  that  when  p.q  G  D  then  Apq  must  be  1,  and  it  can  be 
0  otherwise.  Together  these  guarantee  that  Apq  is  1  when  both  p,  q  G  /A  and  0  otherwise. 
Now  we  can  constrain  the  total  flow  on  ( p ,  q)  to  be  at  most  wpqApq. 

MILP  2  solves  MaxSG-NA  by  extending  MILP  1  for  flow  allocation  to  include  the  network 
augmentation  constraints  of  Equations  (3.1)  and  (3.2)  as  described  above.  The  objective 
in  Equation  (3.3)  is  to  maximize  the  number  of  assigned  groups,  while  Equation  (3.15) 
limits  the  number  of  supplemental  nodes  that  can  be  used  to  at  h.  The  constraint  in  Equa¬ 
tion  (3.1)  is  captured  by  Equation  (3.13),  while  the  constraint  in  Equation  (3.2)  is  captured 
by  Equations  (3.7)  -  (3.12). 

MILP  3  modifies  MILP  2  to  solve  the  MinDep  problem  instead.  The  objective  in  Equa¬ 
tion  (3.20)  is  to  minimize  the  number  of  supplemental  nodes  deployed.  The  MaxSG-NA 
constraint  in  Equation  (3.15)  limiting  the  deployment  size  is  removed  from  MILP  3,  and 
the  MaxSG-NA  constraint  in  Equation  (3.14)  limiting  at  most  one  sink  to  be  assigned  to 
each  group  is  changed  to  the  constraint  that  each  group  must  have  exactly  one  sink  as- 
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MILP  2  MILP  formulation  of  MaxSG-NA. 
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signed  to  it  as  shown  in  Equation  (3.31),  so  that  all  groups  are  satisfied  in  the  MinDep 
problem. 

The  number  of  variables  and  constraints  in  both  MILP  2  and  MILP  3  is  polynomial  in 
the  size  of  the  potential  network  G' .  However,  because  mixed  integer  programming  is 
NP-hard,  there  is  no  known  polynomial-time  algorithm  for  solving  these  formulations  and 
running  times  are  likely  to  be  exponential  in  general. 


3.3.2  Heuristic  Algorithms 

To  help  cope  with  the  exponential  running  times  of  solving  MaxSG-NA  and  MinDep  opti¬ 
mally,  we  develop  heuristic  algorithms  that  iterate  through  the  groups  and  deploy  supple¬ 
mental  nodes  as  necessary  to  satisfy  the  groups. 

Most  of  the  work  is  done  by  the  subroutine  SolveGroup,  shown  in  Algorithm  2.  This 
extends  a  partial  solution  by  finding  a  minimum  deployment  needed  to  satisfy  a  group  S3 
given  the  partial  solution  for  previously  considered  groups  F\{Sj}.  It  does  this  by  solving 
a  modified  version  of  the  MinDep  MILP  but  with  the  following  changes: 

1.  Variables  are  only  solved  for  the  groups  in  F,  with  constraints  updated  accordingly. 

2.  A  new  constraint  to  extend  the  previous  deployment:  D'p  =  1  for  all  p  £  D  (Equa¬ 
tion  (3.49)). 

3.  A  new  constraint  to  extend  the  previous  sink  assignment:  Xlt  =  1  for  all  S',  e  F 
where  Xi  =  t  (Equation  (3.48)). 

The  basic  iterative  heuristic  for  MinDep  is  shown  in  Algorithm  3.  The  while  loop  in 
lines  4-7  iterates  through  the  source  groups.  In  each  iteration,  one  of  the  groups  that 
has  not  yet  been  processed  is  selected  (line  5)  and  the  current  solution  extended  by  calling 
SolveGroup  (line  7). 

We  consider  three  variations  on  this  heuristic  that  depends  on  the  choice  of  the  next  group 
to  process  in  line  5: 

1 .  MD-Rand-H.  The  next  group  is  chosen  at  random. 

2.  MD-MF-H.  The  group  that  requires  largest  increase  in  the  deployment  size  is  chosen 
next.  The  intuition  behind  this  “most  first”  heuristic  is  that  this  group  will  only 
become  harder  to  satisfy  on  future  iterations  and  so  it  should  be  satisfied  as  soon  as 
possible. 

3.  MD-LF-H.  The  group  that  requires  the  smallest  increase  in  the  deployment  size  is 
chosen  next.  The  intuition  behind  this  “least  first”  heuristic  is  that  greedily  choos- 
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MILP  3  MILP  formulation  of  MinDep. 
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Algorithm  2  SolveGroup 

Input:  MaxSG  Instance  X,  current  group  index  j,  set  of  processed  groups  F,  current 
deployment  D,  current  sink  assignments  x 
Output:  Extended  deployment  D' ,  extended  sink  assignment  x' ,  flow  / 

1:  Solve  MILP: 


Minimize  D' 
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2:  Set  D'  4r-  {p  G  P  :  D'p  =  1} 

3:  Set  x'  with  x1  4-  x  and  x'3  G-  t  for  X:)t  =  1 
4:  return  (D',x',f)  41 


Algorithm  3  MD-Iter-H:  Basic  iterative  heuristic  for  MinDep. 

Input:  Instance  X  =  (G,  G' ,  w,  &,  b,  h ):  initial  network  G  =  (V.  E ),  potential  network 
G'  =  (V  UPXU  EP),  capacities  w,  sinks  T,  groups  &  =  {^i, . . .  ,  S'*-},  source 
requirements  b. 

Output:  Solution  O  =  ( D ,  x,  /):  deployment  D,  sink  assignments  x,  and  flows  /. 

D  i —  0  (set  of  deployed  positions) 

Initialize  x  as  empty  assignment 
F  i —  0  (set  of  processed  groups) 
while  F  ^  S  do 
Choose  Sj  e  &  \F 
F  <-  FU{Sj] 

( D,x,f )  SolveGroup(Z,  j,  F,  D,x) 

return  ( D,x,f ) 


ing  the  smallest  increase  in  deployment  size  is  a  good  way  to  minimize  the  total 
deployment  size. 

Algorithm  4  is  the  basic  iterative  heuristic  for  MaxSG-NA.  It  is  identical  to  the  heuristic 
for  MinDep,  except  that  on  each  iteration  it  only  extends  the  solution  if  this  does  not 
exceed  the  limit  on  the  number  of  supplemental  nodes  (lines  8  -  9).  As  with  MD-Iter-H, 
we  consider  three  variations  of  SG-Iter-H  depending  on  the  choice  in  line  5:  SG-Rand-H, 
SG-MF-H,  and  SG-LF-H. 


3.4  Experiments 

In  this  section  we  present  results  of  empirical  evaluation  of  our  algorithms.  We  randomly 
generated  input  instances  and  ran  the  algorithms  on  them.  Each  data  point  reported  here  is 
the  average  over  50  input  instances. 

Input  instances  were  generated  based  on  a  disk  graph  model.  In  the  results  presented 
here,  100  initial  nodes  and  64  potential  locations  were  uniformly  distributed  over  a  square 
region  of  the  plane  measuring  250  x  250.  All  initial  nodes  had  a  range  of  25,  and  all 
supplemental  nodes  had  a  range  of  50.  The  initial  network  G  =  (  V.  E )  was  formed 
by  adding  an  edge  between  two  initial  nodes  if  the  distance  between  them  was  at  most 
the  range  (25).  The  bandwidth  of  all  edges  in  E  was  10.  The  potential  network  G'  = 
(V  U  P,  E  U  EP)  was  formed  by  adding  an  edge  between  a  potential  location  and  another 
node  (initial  node  or  another  potential  location)  whenever  the  distance  between  them  was 
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Algorithm  4  SG-Iter-H:  Basic  iterative  heuristic  for  MaxSG-NA. 

Input:  Instance  X  =  (G,G' h):  initial  network  G  =  (V.  E),  potential  network 
G'  =  (V  U  P,  E  Li  EP ),  capacities  w,  sinks  T,  groups  &  =  {^i, . . . ,  SK},  source 
requirements  b,  number  of  supplemental  nodes  h. 

Output:  Solution  O  =  ( D ,  x,  /):  deployment  D,  sink  assignments  x,  and  flows  /. 

1:  D  •*—  0  (set  of  deployed  positions) 

2:  Initialize  x  as  empty  assignment 
3:  F  -t—  0  (set  of  processed  groups) 

4:  while  F  ^  S  do 
5:  Choose  Sj  e  &  \F 

6:  {£,■} 

7:  (. D x',  /')  F-  SolveGroup(X,  j,  F,  D,  x) 

8:  if  | Z)'  |  <  /z  then 

9:  D  4r-  D',X  <-  x'J  4r-  f 

10:  return  ( D,x,f ) 


at  most  the  supplemental  node  range  (50).  This  reflects  cases  where  supplemental  nodes 
have  longer  ranges  due  to  factors  such  as  higher-gain  antennas  or  environmental  factors 
less  path  loss  due  to  reflection  and  obstructions  between  airborne  supplemental  nodes.  The 
capacity  of  all  edges  in  EP  was  50. 

Group  sizes  were  independently  and  uniformly  distributed  between  2  and  7,  inclusive,  and 
each  input  instance  could  have  groups  of  different  sizes.  Group  membership  was  chosen 
independently  and  uniformly  at  random  from  all  possible  groups  of  the  appropriate  size. 

The  source  requirements  of  each  group  member  was  a  real  number  independently  and 
uniformly  distributed  between  10  and  30.  Deployments  up  to  a  maximum  size  of  4  sup¬ 
plemental  nodes  were  considered  in  these  experiments. 

To  solve  the  MILPs,  we  used  CPLEX  10.0,  a  commercial  solver,  running  on  a  computer 
with  a  3  GHz  Pentium  IV  processor  and  1  GB  of  RAM. 

We  solved  MinDep  problems  both  optimally  using  the  exact  MILP  formulation  and  heuris- 
tically  using  MD-Rand-H,  MD-MF-H,  and  MD-LF-H.  All  algorithms  were  run  for  a  max¬ 
imum  of  2  hours  on  each  instance.  The  average  deployment  sizes  found  by  the  optimal 
and  heuristic  algorithms  for  1  to  6  groups  is  plotted  in  Figure  3.1.  The  three  heuristics 
found  deployments  of  similar  sizes,  although  the  greedy  heuristics  significantly  outper¬ 
formed  the  random  heuristic  for  larger  numbers  of  groups.  The  optimal  algorithm  found 
significantly  smaller  deployments  than  the  heuristics,  with  the  difference  in  deployment 
sizes  increasing  with  the  number  of  groups.  However,  even  for  6  groups,  MD-Rand-H 
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□  Optimal  ■  MD-Rand-H  E  MD-MF-H  □  MD-LF-H 
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Figure  3.1:  Average  deployment  sizes  found  by  Algorithm  3 


found  solutions  within  25%  of  optimal  on  average,  while  the  MD-MF-H  and  MD-LF-H 
found  solutions  within  14%  of  optimal  on  average. 

The  improvement  in  MinDep  solution  quality  of  the  optimal  algorithm  over  the  heuristics 
comes  at  the  cost  of  increased  running  time.  The  average  running  times  are  shown  in  Fig¬ 
ure  3.2,  along  with  the  standard  errors.  While  the  running  times  of  the  optimal  algorithm 
and  the  heuristics  are  very  close  for  1  or  2  groups,  they  quickly  diverge  for  larger  number 
of  groups  as  the  running  time  of  the  optimal  algorithm  increases  dramatically.  The  error 
bars  show  that  the  variability  in  running  times  of  the  optimal  algorithm  also  increases  with 
the  number  of  groups,  while  the  iterative  heuristic  exhibits  less  variability  that  increases 
more  slowly  than  the  optimal  algorithm.  This  suggests  that  heuristics  may  be  better  suited 
in  cases  where  the  highly  variable  running  times  is  undesirable. 

The  running  time  of  the  optimal  algorithm  increases  with  the  number  of  groups,  and  this 
rate  increases  from  1  to  5  groups,  then  seems  to  decrease  from  5  to  6  groups.  However, 
this  is  an  artifact  resulting  from  the  maximum  cut-off  time  of  2  hours,  which  introduces 
an  artificial  cap  on  the  running  time  of  MILP  3.  As  the  number  of  groups  increases,  the 
proportion  of  instances  that  can  be  solved  optimally  within  2  hours  decreases  from  100% 
for  1  and  2  groups  down  to  about  50%  for  6  groups,  as  shown  in  Figure  3.3.  The  heuristics 
all  terminated  within  2  hours  and  so  are  not  plotted  on  that  figure. 

We  also  compared  the  SG-LF-H  to  the  optimal  solution  computed  by  MILP  2  for  1  to  5 
groups.  Figure  3.4  shows  the  number  of  groups  satisfied  by  deployments  of  up  to  4  sup¬ 
plemental  nodes  found  by  SG-LF-H,  normalized  to  the  optimal  number  of  groups  satisfied 
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□  Optimal  ■  MD-Rand-H  □  MD-MF-H  □  MD-LF-H 
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Figure  3.2:  Running  times  for  solving  MinDep  using  MILP  3  and  Algorithm  3. 
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Figure  3.3:  Proportion  of  MinDep  instances  solved  optimally  by  MILP  3  within  two  hours. 
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Figure  3.4:  Average  number  of  groups  satisfied  by  Algorithm  4  as  a  percentage  of  the 
optimal  found  by  MILP  2. 


as  determined  by  MILP  2.  As  can  be  seen,  SG-LF-H  initially  performs  well,  but  solution 
quality  decreases  relative  to  the  optimal  as  the  number  of  groups  increases.  A  comparison 
of  the  running  times  of  the  optimal  MILP  and  SG-LF-H  is  given  in  Figure  3.5.  Both  algo¬ 
rithms  require  similar  amounts  of  time  for  problems  with  1  and  2  groups,  but  the  running 
time  of  the  optimal  algorithm  for  increases  sharply  thereafter,  while  the  running  time  of 
SG-LF-H  increases  much  more  slowly. 
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Figure  3.5:  Running  times  for  solving  MaxSG-NA  using  Mil  .P  2  and  SG-LF-H. 
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Chapter  4 

Flow  Allocation  in  Adversarial 
Environments:  Zero-Sum  Games 


In  the  previous  chapter  we  saw  how  the  agents  could  augment  the  environment  in  order 
to  improve  the  quality  of  flow  allocation  for  task  execution  by  improving  connectivity  and 
reducing  capacity  bottlenecks.  In  this  chapter  we  shift  from  the  capacitated  environments 
examined  in  the  previous  chapters  and  consider  costly  environments,  where  there  is  no 
limit  on  the  amount  of  flow  that  can  be  transmitted  on  each  edge,  but  flows  instead  incur 
costs  proportional  to  their  amounts.  We  shift  our  attention  also  from  environments  that  can 
be  changed  by  the  agents  to  improve  performance,  to  one  that  can  be  changed  by  a  hostile 
adversary  to  decrease  the  performance  of  the  agents  by  increasing  their  costs. 

We  start  by  considering  the  extreme  case  where  the  interests  of  the  agents  and  the  adver¬ 
sary  are  diametrically  opposed.  In  this  case  the  adversary  is  truly  malicious  because  its 
only  interest  is  to  impose  greater  costs  on  the  agents.  For  example,  in  an  urban  robot  set¬ 
ting  robots  can  avoid  locations  that  might  be  observed  by  cameras  placed  by  the  adversary, 
but  doing  so  causes  them  to  take  more  circuitous  routes  and  thus  suffer  higher  travel  costs. 
The  robots  balance  the  tradeoff  between  not  being  observed  with  the  higher  travel  costs. 
A  truly  malicious  adversary  does  not  merely  care  about  observing  the  robots  but  also  the 
travel  costs  of  the  robots,  and  is  only  interested  in  forcing  the  robots’  tradeoff  to  be  as  bad 
as  possible,  even  if  the  robots’  best  course  of  action  ends  up  avoiding  observation  by  the 
adversary  completely.  This  kind  of  strictly  competitive  payoffs  arises  very  naturally  in  the 
sensor  network  domain  where  the  adversary  jams  nodes  in  order  to  force  them  to  use  more 
precious  battery  power  in  stronger  transmissions  or  a  greater  number  of  retransmissions. 
The  sensor  nodes  want  to  minimize  total  battery  depletion  while  the  adversary  seeks  to 
maximize  it,  and  it  doesn’t  matter  if  this  battery  depletion  occurs  because  nodes  transmit 
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through  the  jamming  or  around  it. 

This  strictly  competitive  setting  is  naturally  modeled  as  a  two-player  zero-sum  game 
played  on  a  graph.  We  assume  that  the  agents  work  together  cooperatively,  and  there¬ 
fore  represent  them  by  a  single  player,  called  the  sender.  The  other  player  is  the  adversary. 
In  this  chapter  we  formally  describe  two  zero-sum  games. 

In  the  path  game,  the  sender’s  pure  strategies  are  combinations  of  paths  (unsplittable  flows) 
from  multiple  known  starting  locations  to  a  common,  known  destination.  In  the  multirobot 
domain,  these  represent  paths  taken  by  the  robots  through  the  environment.  The  adversary 
chooses  a  subset  of  attacks  to  play  from  a  set  of  possible  attacks.  In  the  multirobot  domain 
where  the  adversary  has  k  cameras,  each  attack  corresponds  to  the  placement  of  a  single 
camera,  while  a  pure  strategy  is  a  particular  deployment  of  k  cameras  to  locations  in  the 
environment  to  observe  the  robots’  movements.  The  payoffs  for  the  players  are  quantified 
by  the  harm  suffered  by  the  sender  and  is  computed  by  a  harm  function  mapping  strategy 
profiles  to  harm.  Although  zero-sum  games  can  be  solved  in  polynomial  time,  this  is 
polynomial  in  size  of  the  pure  strategy  spaces.  In  the  path  game,  the  strategy  space  of  the 
sender  can  be  exponential  in  the  size  of  the  graph,  while  the  strategy  space  of  the  adversary 
may  be  exponential  in  the  number  of  simultaneous  attacks.  This  makes  a  direct  application 
of  the  traditional  linear  programming  techniques  for  zero-sum  games  impractical  for  the 
path  game. 

The  network  flow  game  addresses  this  difficulty.  This  game  differs  from  the  path  game  in 
the  sender’s  strategy  space.  Rather  than  choosing  discrete  paths,  the  sender  chooses  divis¬ 
ible  network  flows  from  source  nodes  to  a  common  sink  node.  This  naturally  represents 
domains  where  the  quantity  moving  through  the  graph  can  be  divided.  For  example,  in 
wireless  sensor  networks  traffic  can  be  divided  over  multiple  paths.  We  further  show  in 
Section  4.3  that  the  network  flow  game  is  equivalent  to  the  path  game  for  a  class  of  harm 
functions  that  can  be  represented  using  harm  matrices.  Intuitively,  the  attacks  for  these 
harm  functions  correspond  to  a  set  of  costs  on  the  edges  of  the  graph,  with  the  harm  for 
multiple  attacks  additively  combined.  In  such  cases  the  network  flows  represent  marginal 
probabilities  of  edges  being  included  in  mixed  sender  strategies  of  the  path  game.  By 
using  a  compact  representation  based  on  the  marginal  probabilities  of  sender  and  adver¬ 
sary  strategies,  rather  than  full  probability  distributions  over  the  pure  strategies  of  the  path 
game,  we  are  able  to  find  equilibria  in  polynomial  time.  We  describe  a  simple  technique 
for  sampling  a  pure  strategy  for  the  path  game  from  the  network  flow  representation.  In  the 
next  chapter  we  show  how  this  approach  can  be  leveraged  to  solve  non-zero  sum  games 
that  include  both  attack  costs  and  movement  costs. 

A  summary  of  the  notation  used  in  this  chapter  is  provided  in  Table  4.1.  Portions  of  the 
work  in  this  chapter  was  originally  published  in  Okamoto,  et  al.  [47]. 
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Symbol  Description 


A  set  of  possible  attacks  for  the  adversary 

A  adversary  strategy  space,  {A'  C  A  :  \A'\  <  k} 

bs  source  requirement  of  s  G  S' 

A(-)  the  categorical  probability  distribution  over  its  argument,  a  finite  set 

E  set  of  edges 

/  sender  strategy,  a  flow  with  fuv  being  the  amount  of  flow  on  edge  (u,  v )  G 

E 

T  zero-sum  network  flow  game  sender  strategy  space,  the  set  of  feasible 

flows  from  S  to  t 

G  =  (V,  E)  network  with  nodes  V  and  edges  E 
"Hpo  harm  function  for  the  zero-sum  path  game 

k  number  of  attacks  that  the  adversary  can  play  simultaneously 

m  number  of  edges 

M  harm  matrix  with  MtJ  the  harm  from  sending  1  unit  of  flow  on  t3  G  E 

when  the  adversary  plays  a,  G  A 
n  number  of  nodes 

V  zero-sum  path  game  sender  strategy  space,  V\  x  V-2  x  . . .  V\s\ 

Vi  set  of  all  paths  from  .s-t  G  S  to  t 

q  an  adversary  mixed  strategy,  often  expressed  as  a  vector  of  marginal  prob¬ 

abilities  over  A 

S  set  of  source  nodes 

t  the  sink 

V  set  of  nodes 

G  A  adversary  pure  strategy,  a  set  of  attacks 


Table  4.1:  Summary  of  notation  for  Chapter  4 
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4.1  Zero-Sum  Path  Game  (ZS-PG) 


We  first  consider  the  normal  form  of  the  zero-sum  game.  The  zero-sum  path  game  (ZS- 
PG)  is  played  between  a  sender  and  an  adversary  taking  actions  on  a  directed1  graph 
G  =  (V,  E)  with  n  —  \V\  nodes  and  m  =  \E\  edges.  A  subset  of  the  nodes  S  C  V  are  the 
source  nodes  while  another  of  the  nodes  t  e  V  \  S  is  the  sink.  Without  loss  of  generality, 
we  assume  that  there  are  no  incoming  edges  to  any  source  node.  (If  there  is  such  a  source 
node  s,  add  a  new  source  node  s'  to  S  and  an  outgoing  edge  (s',  s )  to  E,  and  remove  s 
from  S.)  Each  source  node  has  a  weight  bs  >  0  representing  the  value  of  that  source  node. 
The  sender  chooses  a  path  ns  from  each  source  s  E  S  to  the  sink  t  (we  assume  that  such  a 
path  exists  for  every  s  E  S),  and  his  pure  strategy  space  is  set  of  all  combinations  of  paths 
from  S  tot,  V. 

The  adversary  has  a  finite  set  of  attacks  A,  and  can  carry  out  up  to  k  attacks  from  A 
simultaneously.  Thus  the  adversary’s  set  of  pure  strategies  A  is  the  set  of  all  subsets 
of  A  of  size  at  most  k,  which  has  size  @(|A|fc).  We  assume  that  both  players  have  full 
knowledge  of  G,  S,  t.  A,  and  k. 

The  payoff  in  this  game  is  quantified  by  the  harm  suffered  by  the  sender  as  a  result  of  one 
or  more  of  his  paths  being  attacked.  As  a  zero-sum  game,  we  assume  that  the  sender  seeks 
to  minimize  the  harm  suffered,  while  the  adversary  seeks  to  maximize  the  harm  inflicted. 
In  general,  harm  may  be  an  arbitrary  function  T-LPG  :  [0,  +oo)  mapping  from  the 

sender’s  choice  of  paths  and  the  adversary’s  choice  of  attacks  to  a  non-negative  number. 
For  convenience,  we  overload  the  notation  to  extend  the  harm  function  to  mixed  strategies, 
with 


^pg  (p,  q )  = 

tt&V  Ce.4 

representing  the  expected  harm  when  the  sender  plays  mixed  strategy  p  E  A('P)  and  the 
adversary  plays  mixed  strategy  q  E  A(*4). 

We  can  now  give  the  normal  form  of  the  zero-sum  path  game. 

Definition  1.  The  zero-sum  path  game  is  a  game  (2,  (V,  A),  (— 'Hpg,  'Hpg))  where 

•  V  =  Vi  x  V2  x  •  •  •  x  V\s\  with  V,  the  set  of  all  paths  from  s*  <G  S  to  t. 

•  A  =  {A1  :  A'  C  A  A  \A!\  <  k}. 

•  "Hpg  is  the  harm  function. 


'The  model  and  algorithms  can  be  applied  to  undirected  graphs  in  a  straightforward  manner. 
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We  are  interested  in  finding  a  mixed  strategy  Nash  equilibrium  strategy  profile  for  the 
sender  and  adversary.  Because  this  is  a  zero-sum  game,  this  corresponds  to  the  minimax 
and  maximin  strategies.  The  sender’s  minimax  problem  is  shown  below,  where  p  €  A('P) 
and  q  e  A  (.4). 


min  max  "Hpg(p,  q) 

p  q 

(4.1) 

subject  to 

Y,p-  =  l 

(4.2) 

ir&V 

(4.3) 

C&A 

Pit  >  0 

VneV 

(4.4) 

QC  >  0 

V(eA 

(4.5) 

We  can  use  the  well-known  linear  programming  techniques  for  finding  such  strategy  pro¬ 
files  for  zero-sum  games  in  time  polynomial  in  the  size  of  V  and  A,  but  these  are  prob¬ 
lematic  because  V  is  exponential  in  n  and  \S\,  and  \A\  =  (^)  =  Q(nk)  in  the  worst 
case. 

Instead,  we  exploit  the  structure  of  a  broad  class  of  harm  functions  to  decompose  the  pay¬ 
off  function  into  a  polynomially-sized  representation,  called  a  harm  matrix.  This  reduces 
the  computational  time  of  finding  equilibria,  while  still  being  able  to  represent  the  payoffs 
for  a  wide  variety  of  harm  functions  that  have  not  been  considered  using  other  representa¬ 
tions.  Although  the  algorithm  we  describe  here  is  specific  to  the  zero-sum  game  of  strictly 
competitive  payoffs,  in  Chapter  5  we  will  leverage  the  network  flow  approach  to  solving 
certain  kinds  of  non-zero-sum  games  for  different  types  of  harm  matrices  that  we  present 
in  the  next  section. 


4.2  Harm  matrices 

Harm  matrices  are  applicable  when  the  harm  function  can  be  decomposed  so  that  harm 
can  be  computed  independently  for  each  pair  of  edge  and  attack,  then  summed  to  calculate 
the  total  harm.  The  harm  matrix  M  is  a  matrix  with  n  rows  and  m  columns,  where 
is  the  amount  of  harm  suffered  by  the  sender  if  the  adversary  plays  attack  a*  e  A  and  the 
sender  chooses  a  path  with  edge  e,-.  Intuitively,  each  attack  specifies  non-negative  weights 
on  the  edges,  the  length  of  an  edge  is  the  sum  of  the  weights  from  the  attacks,  and  harm  is 
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the  sum  of  the  lengths  of  the  paths.  The  following  provides  the  precise  conditions  required 
of  the  harm  function  decomposition. 

Let  7r  e  V  be  a  pure  strategy  of  the  sender  with  ns  denoting  the  path  from  source  node 
s  to  t,  and  let  C  G  A  be  a  pure  strategy  of  the  adversary.  Many  interesting  and  realistic 
harm  functions  can  be  decomposed  in  the  following  way.  First,  the  total  harm  is  the  sum 
of  harm  for  the  individual  paths  from  each  source  to  the  sink: 

■Hpg(7T,  C)  =  ^  h^8'  0  (4-6) 

seS 


Second,  the  harm  for  a  set  of  attacks  is  the  sum  of  harm  for  the  individual  attacks  in  (: 

h(ns,C)  ='^2h(7rs,ci)  (4.7) 

Finally,  the  harm  for  ns  is  the  sum  of  the  harm  for  the  individual  edges  in  ns,  and  this  harm 
is  a  linear  function  of  the  weight  of  s,  with  the  specific  linear  function  depending  on  the 
edge  e  and  attack  a  through  a  constant  value  aae : 

h(irs,a)  =  ^2  aaebs.  (4.8) 

e£irs 


For  harm  functions  that  satisfy  these  properties,  we  construct  a  harm  matrix  M  with  /I 
rows  and  m  edges  where  entry  =  aaiej  ■ 

We  now  turn  our  attention  to  several  interesting  harm  functions  and  their  harm  matrix 
representation.  For  the  purposes  of  exposition  we  consider  a  radio  jamming  attack,  in 
which  the  adversary  transmits  radio  signals  to  disrupt  the  sender’s  communications.  This 
attack  increases  latency  because  packets  need  to  be  retransmitted.  If  multiple  points  on 
a  pathway  are  disrupted  by  the  attack,  additional  latency  is  suffered,  resulting  in  additive 
harm. 

Consider  the  simple  path  intersection  harm  function,  where  uniform  harm  is  suffered  if 
and  only  if  a  node  on  the  pathway  is  attacked,  as  arises  if  the  adversary  must  directly  jam 
a  node  on  a  pathway  to  disrupt  communication,  and  each  attack  causes  the  same  increase 
in  latency.  In  this  case  the  attacks  correspond  to  nodes  in  V  and  the  harm  is  represented 
by  the  following  harm  matrix: 


71^-path  int 
Mij 


1  if  6j  =  (u,  Vi)  for  some  node  u  G  V 
0  otherwise 


(4.9) 
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More  generally,  different  nodes  may  be  more  or  less  susceptible  to  jamming,  leading  to 
different  amounts  of  harm  being  suffered  when  the  sender  routes  through  an  attacked  node. 
This  is  represented  by  a  harm  matrix  with  heterogeneous  values: 


7i  //-gen  path  int 

Mij 


Ci  if  ej  =  (it,  Vi)  for  some  node  u  G  V 
0  otherwise 


(4.10) 


Jamming  is  not  the  only  cause  of  latency.  Latency  is  also  incurred  on  every  transmission 
on  an  edge  on  the  path  chosen  by  the  sender.  This  is  an  example  of  costly  transmission  in 
which  the  sender  incurs  a  cost  every  time  he  utilizes  an  edge,  irrespective  of  which  nodes 
the  adversary  has  attacked;  examples  include  latency  or  battery  power  usage.  By  using 
a  harm  matrix  that  includes  both  the  harm  suffered  from  the  adversary’s  attacks  and  the 
transmission  costs,  the  sender  can  rationally  reason  about  the  tradeoff  between  them.  Such 
a  harm  matrix  may  use  homogeneous  cost  values  (biasing  toward  pathways  with  fewer 
hops)  or  heterogeneous  cost  values  that  depend  on  the  edge,  representing  characteristics 
such  as  requiring  more  battery  power  to  transmit  to  more  distant  nodes  or  to  nodes  in  high 
noise  areas.  In  the  following  harm  matrix,  harm  Cj  (cost  of  transmission)  is  incurred  for 
every  unit  of  flow  transmitted  on  edge  ej. 


Ml 


1  +  Cj  if  ej  =  («,  Vi)  for  some  node  u  G  V 
Cj  otherwise 


(4.11) 


There  is  also  no  requirement  that  paths  must  intersect  the  attacked  nodes  for  harm  to  be 
suffered.  In  the  jamming  example,  attacking  a  node  in  a  wireless  network  may  also  jam 
neighboring  nodes,  even  if  those  nodes  are  not  communicating  with  the  attacked  node. 
Thus  transmissions  on  all  edges  to  neighbors  of  an  attacked  node  incur  increased  latency. 


M1 

u 


nonlocal 


Cj  if  ej  =  ( u ,  v)  and  (v,  ly)  G  E 

0  otherwise 


(4.12) 


In  then  next  subsection  we  describe  the  network  flow  game  that  explicitly  uses  the  harm 
matrix  structure  along  with  marginal  probability  distributions  to  efficiently  represent  the 
strategy  spaces  of  the  sender  and  adversary. 
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4.3  Zero-Sum  Network  Flow  Game  (ZS-NFG) 


In  the  zero-sum  network  flow  game,  the  sender  chooses  flows  from  the  source  nodes  to 
the  sink,  sending  bs  units  of  flow  from  each  source  s  G  S'  to  the  sink  t.  The  flows  are 
represented  by  an  m  x  1  column  vector  /,  where  fuv  is  the  amount  of  flow  sent  on  edge 
(u,  v);  when  denoting  an  edge  simply  as  e  G  E,  we  also  use  the  notation  fe.  The  sender’s 
strategy  space  T  is  the  set  of  all  feasible  flows  from  the  source  nodes  to  the  sink,  that  is, 
all  /  such  that 


fvu  =  K  +  51  VveV\{t}  (4.13) 

(v,u)gE  (u,v)eE 

fuv>  0  V(u,v)eE.  (4.14) 

Equation  4.13  is  the  flow  conservation  constraint  requiring  outgoing  flow  for  all  nodes 
(other  than  the  sink)  to  be  equal  to  the  source  requirement  of  the  node  plus  the  sum  of 
the  incoming  flow.  Equation  (4.14)  ensures  that  the  flow  on  each  edge  is  non-negative2. 
Flows  may  be  divided  on  alternate  paths  from  the  source  nodes  to  the  sink,  leading  to  a 
continuous  strategy  space  for  the  sender  if  there  are  at  least  two  paths  from  any  source 
node  to  the  sink.  The  adversary’s  strategies  in  ZS-NFG  are  exactly  the  same  as  those  in 
ZS-PG,  with  his  pure  strategy  set  A  being  all  subsets  of  A  of  up  to  k  attacks,  for  a  total  of 
@(|3L|fc)  pure  strategies. 

The  payoffs  in  ZS-NFG  are  quantified  by  the  harm  as  in  ZS-PG,  but  in  ZS-NFG  we  only 
consider  harm  functions  that  can  be  represented  using  harm  matrices.  For  a  harm  matrix 
M,  sender  pure  strategy  /  G  J,  and  adversary  pure  strategy  (,'  G  A  the  harm  is 

f/(/,C)  =  5>wa[M]/  (4.15) 

as  C 

where  rowa[M]  is  the  1  x  m  row  vector  corresponding  to  the  row  of  M  for  attack  a  G  A. 
For  convenience,  we  also  overload  this  notation  to  finite  mixed  strategies  of  the  sender  and 
adversary,  with  the  expected  harm 

H(p,  q )  = 

feF  ce.4 


2Other  network  flow  formulations  sometimes  represent  the  net  flow  between  nodes,  which  requires  that 
fUv  =  —fvu-  Our  formulation  instead  uses  the  actual,  non-negative  amount  of  flow  sent  on  each  directed 
edge. 
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where  p  £  A (F)  for  finite  F  C  F  and  q  £  A  (^4). 

We  are  interested  in  finding  Nash  equilibrium,  which,  because  of  the  zero-sum  payoffs,  are 
strategy  profiles  where  the  sender  plays  a  strategy  that  minimizes  the  maximum  harm  (i.e., 
the  sender  plays  a  minimax  strategy)  and  the  adversary  plays  a  strategy  that  maximizes 
the  minimum  harm  (i.e.,  the  adversary  plays  a  maximin  strategy).  We  consider  equilibria 
where  the  sender  is  a  playing  a  pure  flow  strategy  (i.e.,  some  /  £  F)  while  allowing  the 
adversary  to  play  a  mixed  strategy  q  £  A  (A).  Before  continuing,  we  must  prove  that  such 
an  equilibrium  must  exist. 

We  first  prove  that  for  any  mixed  sender  strategy  over  a  finite  set  of  flows  there  exists  a 
pure  sender  strategy  with  the  same  expected  payoff  when  played  against  any  adversary 
pure  strategy. 

Lemma  1.  Given  a  finite  set  of  flows  F  C  T ,  a  mixed  sender  strategy  p  £  A  (F),  and  an 
adversary  pure  strategy  (  £  A,  there  exists  a  flow  f  £  T  such  that 


HUX)  =  E  Pf'HU',  C). 

f'eF 

Proof.  We  prove  this  by  construction.  Set  the  amount  of  flow  fe  on  each  edge  e  to  be  the 
expected  amount  of  flow  under  the  mixed  strategy  p.  This  is  easily  shown  using  vector 
addition: 


/  4  !>/-/'  (4.16) 

f'eF 

We  can  now  verify  that  the  payoff  with  /  is  equal  to  the  expected  payoff  with  p. 

H(f,  C)  =  ^  rowa  [M\f 

aG  C 

=  £row a[M] 

aec  f'eF 

=  £^£row«W' 

f'eF  aec 

=  (/'.<)• 

f’eF 

□ 

As  a  corollary,  the  lemma  is  applicable  to  mixed  adversary  strategies,  due  to  the  linearity 


57 


of  expectation  and  the  harm  function. 

Corollary.  Given  a  finite  set  of  flows  F  C  T,  a  mixed  sender  strategy  p  G  A  (F),  and  an 
adversary  mixed  strategy  q  G  A  M),  there  exists  a  flow  f  G  T  such  that 


f'eF 

Theorem  1.  Consider  a  zero-sum  path  game  with  a  harm  function  that  can  be  represented 
by  a  harm  matrix,  and  a  zero-sum  network  flow  game  with  the  same  harm  matrix.  Then 
the  following  must  hold: 

1.  For  all  sender  mixed  strategies  p  G  A("P)  of  the  ZS-PG  there  exists  a  sender  pure 
strategy  f  G  F  of  the  ZS-NFG  such  that  for  all  adversary  mixed  strategies  q  G 
AM),  the  expected  harm  in  the  two  games  is  the  same. 

2.  For  all  sender  pure  strategies  f  G  F  of  the  ZS-NFG  there  exists  a  sender  mixed 
strategy  p  G  AM)  of  the  ZS-PG  such  that  for  all  adversary  mixed  strategies  q  G 
AM),  the  expected  harm  in  the  two  games  is  the  same. 


Proof.  We  prove  the  first  claim  by  construction.  Let  p  G  A('P)  and  q  G  AM).  Recall  the 
definition  of  V  from  Definition  1:  V  —  V\  x  •  •  •  x  V\s\  where  V,  is  the  set  of  all  paths  from 
Si  G  S  to  t.  Without  loss  of  generality  we  restrict  the  V,  to  include  only  simple  paths.  For 
each  path  nSi  G  V,  we  construct  a  corresponding  flow,  f71'  that  sends  the  full  amount  of 
flow  hSi  from  s*  to  t  along  the  edges  of  t tp 


fl 


A 


if  e  G  7Tj 
otherwise 


Ve  G  E.  (4.17) 


For  each  n  =  pi, . . . ,  7r|5|)  G  V,  we  then  construct  a  flow  p  that  sends  the  full  amount  of 
flow  bs  along  "r  for  each  st  G  S.  This  is  achieved  by  summing  the  flows  from  each  source: 

P  =  E f?  VeeE  (4.18) 

SiGS 


We  can  now  define  a  set  F  of  flows  in  the  ZS-NFG  that  correspond  to  the  ZS-PG  pure 
strategies: 


F  A  {p  :  vr  G  V} 


Because  V  is  finite,  F  must  be  finite  as  well.  We  extend  the  ZS-PG  mixed  strategy  p  to  a 
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ZS-NFG  mixed  strategy  p'  G  A (F)  by  setting  p'^  A  pn  for  all  n  G  V.  We  then  show  that 
the  expected  harm  of  (p,  q )  in  the  ZS-PG  is  the  same  as  the  expected  harm  of  (// .  q)  in  the 
ZS-NFG.  The  expected  harm  in  the  ZS-PG  is 

^pg  (p,  <?)  =  Z  Zp^  c  Z Z Z  Miibs 

ttEV  C,€lA  sG5  CjGtts 

=  Z  Z«  Z  Z  Z 

weF  CG.4  “iSC  sG>S  ejETTs 

-  Z  Z^< c  Z  Z 

7tG7^  £E.A 

=  Z  Z «  Z roWa*  tM]-^ 

ttsP  ce.4  aiSC 

=  £Emc  (/'.<) 

?reP  Ce^t 

=  E  Ep'/-«<ff(/'-o 

/^eF  ceyt 


which  is  the  expected  harm  in  the  ZS-NFG. 

By  Lemma  1,  there  exists  a  pure  ZS-NFG  sender  strategy  /  e  T  with  the  same  expected 
harm  as  //,  and  thus  with  the  same  expected  harm  as  p  in  the  ZS-PG.  Thus  the  first  claim 
is  proved. 

We  prove  the  second  claim  by  construction  as  well.  Let  /  G  T  and  q  G  A (^4).  For  each 
7TSi  G  Vi  we  set 


pr(^j  -  n 

(u,v)E  nSi 


fu 


bu  +  Y2(u',u)eE  fu'v 


(4.19) 


and  for  n  G  V  we  set 


Pn=  n  Pr(7r*i)- 

SiES 


(4.20) 

□ 


Theorem  1  establishes  a  very  close  relationship  between  ZS-PG  and  ZS-NFG:  in  essence, 
the  two  games  model  the  same  strategic  interaction.  In  other  words,  ZS-NFG  can  be  seen 
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as  an  alternate  representation  for  the  normal  form  representation  used  by  ZS-PG.  As  we 
will  see,  this  representation  can  be  solved  more  efficiently  than  the  normal  form  of  ZS-PG. 
We  explore  this  connection  further  in  Section  4.5,  but  for  the  moment  we  note  an  important 
corollary  of  Theorem  1,  the  existence  of  an  equilibrium  for  ZS-NFG. 

Corollary.  The  zero-sum  network  flow  game  has  a  Nash  equilibrium  (/,  q)  where  the 
sender  plays  a  pure  strategy  f  G  T  and  the  adversary  plays  a  mixed  strategy  q  G  A  (,4). 

Proof.  This  follows  from  Theorem  1  and  the  existence  of  a  Nash  equilibrium  for  ZS-PG 
as  a  finite  strategy  game.  □ 

Note  that  the  corollary  guarantees  the  existence  of  an  equilibrium  where  the  sender  plays 
a  pure  flow  strategy,  allowing  us  to  restrict  our  search  to  pure  strategies  for  the  sender. 
Although  there  may  be  multiple  equilibria,  the  game  is  zero-sum  and  so  we  know  that 
there  are  no  equilibrium  selection  issues:  the  payoffs  (i.e.,  the  harm)  in  all  equilibria  are 
the  same,  and  the  equilibria  are  all  interchangeable.  Moreover,  we  know  that  the  equilibria 
must  satisfy  the  minimax  conditions:  the  sender’s  flow  must  minimize  the  maximum  harm 
over  all  possible  pure  strategies  played  by  the  adversary,  and  the  adversary’s  mixed  strategy 
must  maximize  the  minimum  harm  over  all  possible  pure  strategies  played  by  the  sender. 
We  can  write  the  sender’s  network  flow  minimax  problem  as 


Minimize  max  > 

fer  Ce-4  ^ 

row  a[M]f 

(4.21) 

subject  to 

^  ^  fvu  bv  -\- 

E  ■/»» 

\/veV\  {t} 

(4.22) 

u:(v,u)£E 

u:(u,v)£E 

fuv>  0 

V(u,  v )  G  E. 

(4.23) 

Equation  (4.22)  is  the  flow  conservation  constraint  requiring  outgoing  flow  for  all  nodes 
(other  than  the  sink)  to  be  equal  to  the  source  requirement  of  the  node  plus  the  sum  of  the 
incoming  flow.  Equation  (4.23)  requires  all  flows  to  be  non-negative. 

It  is  helpful  to  gain  some  intuition  into  the  structure  of  the  network  flow  game  equilibria. 
Assume  that  we  are  using  the  homogeneous  path  intersection  harm  matrix  (so  that  attacks 
correspond  to  nodes).  From  the  sender’s  perspective,  his  choice  /  is  a  best  response  to  the 
adversary  if  he  can’t  improve  it  by  changing  some  of  the  flow  from  one  of  the  current  paths 
to  a  better  path,  i.e.,  a  path  with  lower  probability  of  intersecting  a  node  under  attack.  Note 
that  if  the  sender  is  sending  flow  on  a  path  ns  from  s  G  S  to  t,  and  there  is  a  path  with 
lower  probability  n's  from  s  to  t.  then  he  should  move  all  of  the  flow  from  tt,  to  7 r' .  Let  tt, 
be  a  path  from  one  of  the  s  G  S  to  t,  and  let  f*  denote  the  best  response  amount  of  flow 
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sent  on  7 rs.  We  can  write  the  sender’s  best  response  property: 


/;.>«  =*•  X> 

V£7T 


min 

from  5  to  t 


uGir' 


(4.24) 


Thus,  the  adversary  should  evenly  distribute  probability  among  nodes  so  that  all  paths 
from  one  or  more  source  nodes  to  the  sink  have  equal  probability  of  being  attacked. 

Now  for  the  adversary,  a  distribution  q  is  a  best  response  if  he  can’t  improve  it  by  choosing 
a  different  distribution  q'  with  greater  harm.  Fixing  the  sender’s  strategy  /,  the  adversary’s 
payoff  for  q  is  qMf  and  so  a  rational  adversary  will  choose  to  put  all  of  her  probability  on 
the  nodes  with  the  corresponding  greatest  harm.  Let  ( M f)v  denote  the  element  for  node 
v  in  column  vector  M /,  then 


Pi  >  0  =►  (. Mf)v  =  ma x(Mf)v,  (4.25) 

v'GV 

Hence,  the  sender  should  minimize  the  maximum  ( Mf)v  in  order  to  minimize  the  max¬ 
imum  harm  that  will  be  caused  by  the  adversary.  We  call  the  term  ( Mf)v  the  potential 
harm  because  it  represents  the  amount  of  harm  that  could  be  suffered  if  the  adversary 
attacked  node  v. 

As  a  consequence  of  these  two  properties,  the  sender  will  route  flows  to  distribute  the 
potential  harm  ( M  f)v  as  evenly  as  possible.  When  we  are  considering  the  path  intersection 
harm  matrix  Mpath  int,  the  sender’s  equilibrium  strategy  effectively  performs  network  load 
balancing.  Furthermore,  the  set  of  nodes  with  maximum  (M  f)v  form  a  vertex  cut  in  the 
network  separating  a  subset  of  the  source  nodes  from  the  sink.  While  graph  theoretic 
approaches  (e.g.,  [66])  can  find  such  vertex  cuts  for  the  case  of  a  single  source  and  single 
sink  and  the  simple  path  intersection  harm  function,  they  are  incapable  of  handling  more 
complex  problems.  Our  approach  is  unique  in  balancing  the  potential  harm  rather  than  just 
the  network  flow. 

Because  this  vertex  cut  U  will  contain  the  nodes  with  maximum  (M f)v,  the  adversary  will 
only  attack  nodes  in  the  vertex  cut,  and  by  virtue  of  it  being  a  vertex  cut,  the  adversary  can 
attack  those  nodes  to  guarantee  that  all  paths  from  a  subset  of  source  nodes  S'  C  S  on  the 
other  side  of  the  cut  must  go  through  an  attacked  node.  Given  U ,  there  is  a  minimum  subset 
U'  which  is  still  a  vertex  cut.  U'  has  the  further  property  that  it  contains  no  redundant  nodes 
in  the  sense  that  no  node  can  be  removed  and  still  guarantee  that  all  paths  from  S'  to  t  pass 
through  it.  Thus  the  adversary  will  assign  non-zero  probability  to  the  nodes  in  U' ,  evenly 
divided  so  that  all  paths  have  equal  probability  of  being  attacked. 
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4.4  Computing  Network  Flow  Equilibrium 


In  this  section  we  describe  the  linear  program  for  finding  the  equilibrium  strategies.  When 
the  adversary  can  execute  a  single  attack,  the  sender’s  equilibrium  strategy  can  be  found 
via  a  straightforward  linearization  of  the  sender’s  network  flow  minimax  problem,  LP  1. 
The  harm  is  represented  by  the  variable  H  and  is  constrained  to  be  at  least  as  great  as  the 


LP  1  Network  Flow  LP  for  k  —  1. 

Minimize  H 

f,H 

(4.26) 

subject  to: 

H  >  row a[M]f 

Va  e  A 

(4.27) 

^  '  fvu  by  T  ^  ^  fuv 

\/veV\  {t} 

(4.28) 

(v,u)EE  (u,v)eE 

fuv  >  0 

V(m,  v)  G  E 

(4.29) 

harm  resulting  from  any  single  attack  by  Eq.  (4.27). 

The  term  row a[M]  denotes  the  row 

corresponding  to  attack  a  in  matrix  M.  LP1  has  \E\  +  1  variables  and  2 n  — 
Thus  it  can  be  solved  in  polynomial  time  (with  respect  to  n)  [8]. 

1  constraints. 

We  now  extend  LP  1  to  allow  the  adversary  to  execute  multiple  attacks  simultaneously. 
This  program  introduces  an  additional  zl  variables,  the  Aa.  We  first  observe  that  when 

LP  2  Network  Flow  LP  for  general  k  >  1 

Minimize  kH  +  }  Xa 

f,H,\ 

a&A 

(4.30) 

subject  to: 

^  ]  fvu  by  T"  ^  fuv 

Vu  e  V  \  {t} 

(4.31) 

( v,u)GE  (u,v)€E 

H  >  row a[M]f  -  Xa 

Mae  A 

(4.32) 

fuv  >  0 

M(u,  v)  e  E 

(4.33) 

Aa  >  0 

Mae  A 

(4.34) 
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k  =  1,  LP  2  reduces  back  to  LP  1,  as  expected,  with  all  Aa  =  0.  When  k  >  1,  the  potential 
harm  for  each  attack  a  £  A  is  ( Mf)a  =  row a[M]f.  As  a  best  response  the  adversary  will 
attack  a  set  Z  C  A  of  size  k  that  causes  maximum  harm  by  choosing  the  k  attacks  with 
greatest  potential  harm.  If  we  let  H  denote  the  £;th  largest  potential  harm,  it  follows  that 
the  total  harm  will  be 


row,,  [M}f  =  5>owJ M]f  -  H)  +  H 

CL^Z  CL^Z 

=  kH  +  J](rowa[M]/  -  H). 

a&Z 

LP  2  minimizes  this  total  harm  over  all  possible  such  Z  by  letting  Xn  be  the  variable  for 
row a[M]f  -  H. 

The  adversary’s  equilibrium  strategy  can  be  found  by  solving  LP  3,  the  dual  program  of 
LP  2.  There  are  \A\  of  the  q  variables,  with  qa  for  a  £  A  being  the  marginal  probability 

LP  3  Adversary  Network  Flow  LP  for  general  k  >  1 


Maximize  bsrs 

(4.35) 

s£S 

subject  to: 

ru  ~  rv  <  qT co\u,v)[M] 

V(tt,  v)  £  E,U  7^  t,V  7^  t 

(4.36) 

-  rv  <  qTco\t,v)[M] 

V(t,  v)  £  E 

(4.37) 

r u  <  gTcol (u,t)[M] 

V(m,  t)  £  E 

(4.38) 

5 Zqv  =  k 

(4.39) 

v€V 

0<qv<l 

Vu  £  V 

(4.40) 

of  the  adversary  playing  a  strategy  that  includes  attack  a.  By  Eq.  (4.39),  the  sum  of  these 
marginal  probabilities  must  equal  k  because  the  adversary  can  execute  k  attacks  simulta¬ 
neously.  There  is  a  variable  rv  for  each  v  e  V  \  {t },  representing  the  least  amount  of  harm 
that  the  sender  could  suffer  sending  one  unit  of  flow  starting  from  v.  This  is  constrained 
by  Eq.  (4.36)  which  says  that  for  an  edge  (u.  v )  £  E,  the  least  harm  the  sender  can  suffer 
starting  from  u  can  be  no  more  than  the  harm  suffered  crossing  the  edge  (  u  .  v)  added  to  the 
least  harm  starting  from  v.  The  notation  col(U;„)[M]  denotes  the  column  for  edge  (u.  v)  in 
matrix  M.  The  objective  is  to  maximize  the  harm  starting  from  the  source  nodes,  weighted 
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by  the  amount  of  flow  that  must  be  sent  from  each  source  node. 


4.5  Using  ZS-NFG  to  Solve  ZS-PG 

The  network  flow  game  represents  cases  where  the  sender  is  moving  divisible  flow  through 
the  network.  This  is  applicable  for  cases  where  the  assets  moved  through  the  network 
really  are  divisible,  as  may  be  the  case  with  communication  traffic  or  large  quantities  of 
physical  goods  in  transportation  convoys.  Despite  the  infinite  strategy  space  for  the  sender, 
LP  2  provides  a  way  to  find  an  equilibrium  strategy  in  polynomial  time  with  respect  to  G 
and  A. 

In  addition  to  modeling  cases  of  divisible  flow,  the  zero-sum  network  flow  game  models 
the  strategic  interactions  in  the  zero-sum  path  game  with  a  harm  matrix,  as  shown  in 
Theorem  1:  every  mixed  strategy  p  e  A(P)  in  the  ZS-PG  has  a  corresponding  pure 
strategy  /  e  T  in  the  ZS-NFG  that  has  exactly  the  same  payoff  behavior  against  the 
adversary,  and  vice  versa.  Thus  we  can  use  LP  2  to  find  an  equilibrium  sender  strategy 
for  ZS-NFG,  and  we  know  that  there  is  a  corresponding  equilibrium  sender  strategy  p  for 
ZS-PG. 

To  see  why  the  ZS-NFG  flow-based  representation  is  so  much  more  efficient  than  the  ZS- 
PG  normal  form  representation,  we  turn  to  the  properties  of  harm  functions  that  can  be 
represented  by  harm  matrices,  and  to  the  proof  of  Theorem  1.  By  Equation  (4.6),  the  harm 
for  each  source  node  can  be  computed  independently,  with  the  total  harm  being  the  sum 
over  all  source  nodes.  Likewise,  in  the  proof  of  the  first  claim  of  Theorem  1  for  each  ZS- 
PG  pure  strategy  7r  e  V  we  compute  a  flow  fni  for  each  source  sl  using  Equation  (4.17), 
then  sum  them  to  get  the  total  flow  fn  as  shown  in  Equation  (4.18).  The  amount  of  flow 
fl  on  each  edge  e  is  is  the  number  of  paths  that  include  e,  weighted  by  the  weights  b  of 
the  source  nodes. 

For  a  ZS-PG  mixed  strategy  p  e  V  the  flow  is  the  sum  of  the  flows  for  each  pure  strategy, 
weighted  by  the  probability  of  that  pure  strategy,  as  constructed  by  Equation  (4.16).  This 
means  that  the  amount  of  flow  fe  on  each  edge  e  is  the  expected  value  of  the  number  of 
paths  that  include  e,  weighted  by  the  weights  of  the  source  nodes.  When  b  =  1  this  is 
just  the  expected  number  of  paths  that  include  e,  and  by  linearity  the  component  /*  due  to 
each  source  s  is  the  marginal  probability  that  e  is  on  a  path  played  from  s  to  t.  This  is  the 
key  to  ZS-NFG  providing  an  exponentially  more  efficient  representation  of  the  sender’s 
strategy  than  ZS-PG:  because  of  the  harm  matrix  structure  of  the  harm  function,  it  suffices 
to  represent  only  the  marginal  probabilities  of  edges  being  used,  instead  of  the  full  joint 
probability  distribution  over  all  possible  combinations  of  paths. 
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To  find  a  mixed  strategy  p  e  A(P)  of  ZS-PG  that  corresponds  to  a  flow  /,  we  can  ex¬ 
plicitly  represent  the  probability  of  each  it  €  V  with  a  variable  p^  and  write  a  system  of 
linear  equations  that  relates  the  joint  probability  distribution  p  to  the  marginal  probabil¬ 
ity  distribution  /.  In  general  there  may  be  many  possible  mixed  strategies  of  ZS-PG  that 
correspond  to  each  flow  /,  which  will  result  in  this  system  of  equations  having  multiple 
possible  solutions,  but  all  of  these  solutions  will  have  the  same  payoff  behavior  because 
they  are  all  equilibria.  However,  it  is  obvious  that  solving  for  p  in  this  manner  is  impracti¬ 
cal  as  the  number  of  variables  is  \V\,  which  may  be  exponential  in  the  size  of  G  in  general. 
In  fact,  just  specifying  p  completely  may  require  exponential  time  in  general. 

Thus,  instead  of  explicitly  solving  for  p  given  /,  we  instead  sample  a  pure  strategy  n  e  V 
from  the  distribution  p,  without  ever  computing  p  directly.  The  construction  of  p  in  the 
proof  of  the  second  claim  of  Theorem  1  is  one  specific  mixed  strategy  of  ZS-PG  that 
corresponds  to  /,  and  it  is  easy  to  use  for  sampling.  According  to  Equation  (4.20),  we  can 
sample  paths  from  each  source  node  independently.  This  is  convenient  for  applications,  as 
agents  represented  by  the  source  nodes  do  not  need  to  coordinate  with  each  other  once  they 
are  provided  with  the  flow  /.  The  probability  distribution  of  paths  for  each  source  node  is 
given  by  Equation  (4.19)  and  is  also  very  convenient.  Starting  from  the  source  node,  the 
next  edge  to  take  is  chosen  probabilistically  according  to  the  fraction  of  the  total  outgoing 
flow  of  the  current  node  on  each  edge.  Thus  the  agent  starts  at  the  source  node  and  executes 
a  Markovian  policy  where  the  state  is  the  current  node,  and  the  transition  to  the  next  state 
is  determined  probabilistically  by  the  fraction  of  outgoing  flow  to  each  neighbor.  This 
provides  a  very  natural,  polynomial  time  way  to  operationalize  the  equilibrium  computed 
by  LP  2 

An  adversary  pure  strategy  can  also  be  sampled  from  the  marginal  probabilities  computed 
in  LP3.  This  involves  sampling  sets  of  k  attacks  from  A  and  can  be  accomplished  in 
polynomial  time  by  using  weighted  random  sampling  [17]  with  the  marginal  probabilities 
as  weights,  or  comb  sampling  [62]. 


4.6  Experiments 

4.6.1  Simulation  setup 

We  simulated  a  multiagent  system  in  which  agents  (nodes)  were  distributed  uniformly  at 
random  in  a  50  x  50  region  of  the  plane.  The  agents  could  communicate  using  a  multi-hop 
network,  where  agents  within  a  Euclidean  distance  of  10  were  neighbors  in  the  network. 
This  resulted  in  a  network  with  many  paths  between  any  two  nodes  on  average,  but  where 
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the  graph  is  not  fully  connected  (in  which  case  the  problem  is  trivial).  In  this  section,  the 
number  of  agents  is  denoted  by  n,  the  number  of  source  nodes  by  s,  and  the  number  of 
nodes  that  can  be  attacked  by  the  adversary  by  k.  Each  source  node  had  a  flow  distributed 
uniformly  at  random  between  5  and  20.  Each  point  in  the  figures  is  an  average  over  100 
randomly  generated  instances. 


Results 

We  performed  three  sets  of  experiments  using  the  CPLEX  10.0.1  solver  on  a  Linux  ma¬ 
chine  with  a  2.40  GHz  Intel  Core  2  processor  with  4GB  of  RAM.  The  first  set  of  exper¬ 
iments  present  the  average  harm  found  by  LP  2  as  we  increase  the  network  size  and  the 
number  of  source  nodes.  Figure  4.1  shows  the  network  size  on  the  x-axis  and  the  harm  on 
the  y-axis.  The  5  lines  in  the  graph  correspond  to  the  number  of  source  nodes  s  increasing 
from  1  to  5.  k  was  set  to  1  in  this  experiment.  From  the  graph  we  obtain  that  as  the  number 
of  nodes  in  the  network  increase  and  other  parameters  stay  constant,  the  harm  decreases. 
This  is  expected  because  the  sender  can  spread  the  flow  across  many  more  nodes  thus 
decreasing  the  harm  caused  by  the  adversary  at  any  particular  node.  The  same  trend  is 
observed  across  all  values  of  s. 

For  network  sizes  less  than  500  nodes,  the  harm  increases  with  the  number  of  source  nodes 
because  more  source  nodes  imply  more  flow.  For  network  sizes  greater  than  500,  there  is  a 
very  small  difference  in  the  harm  for  varying  source  nodes  because  the  sender  can  spread 
the  flow  across  a  large  number  of  paths,  thus  decreasing  the  ability  of  the  adversary  to 
cause  harm  at  any  single  node. 

Figure  4.2  studies  the  effect  on  running  times  of  LP  2  as  the  network  size  and  number  of 
source  nodes  increase.  The  number  of  nodes  in  the  network  is  shown  on  the  x-axis  and 
the  average  running  times  in  seconds  is  plotted  on  the  y-axis.  The  5  lines  in  the  figures 
correspond  to  the  various  source  node  settings.  The  plot  shows  that  as  the  number  of  source 
nodes  increase  the  running  time  increases,  but  even  at  1000  nodes  the  running  time  is  on 
the  order  of  a  second.  This  shows  that  LP  2  is  a  fast  algorithm  for  finding  the  equilibrium 
in  large  networks. 

Our  next  experiment  studies  the  solution  quality  and  runtime  results  for  LP  2  for  k  =  1 
and  for  k  between  10  to  50  in  increments  of  10,  as  shown  in  Table  4.2.  Note  that  n  =  600 
and  s  =  3  for  this  experiment.  From  the  table  we  obtain  that  as  k  increases  the  harm 
caused  in  the  network  increases  as  expected.  In  fact,  for  each  jump  of  10  nodes  in  k,  the 
harm  caused  also  increases  fairly  uniformly  i.e.  about  5  units.  However,  the  running  time 
is  unaffected  by  an  increasing  k. 
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Number  of  nodes 


Figure  4.1:  Harm  as  a  function  of  network  size  and  number  of  source  nodes 
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Figure  4.2:  Running  times  for  increasing  network  size  and  source  nodes 
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k 

Harm 

Runtime 

1 

.583 

.344 

10 

5.825 

.343 

20 

11.651 

.337 

30 

16.960 

.339 

40 

21.950 

.338 

50 

26.212 

.373 

Table  4.2:  Effect  of  varying  k  on  harm  and  runtime. 


Our  last  experiment  highlights  ability  of  LP  2  to  balance  harm  and  network  performance. 
We  evaluated  LP  2  with  the  costly  communication  harm  matrix,  and  measured  the  average 
number  of  hops  in  the  resulting  pathways.  We  compared  to  the  RANGER  algorithm  [62], 
an  algorithm  that  maximizes  security  but  does  not  take  network  performance  into  account, 
and  a  shortest  paths  (SP)  algorithm  that  optimizes  for  network  performance  without  regard 
for  security.  The  results  are  shown  in  Figure  4.3,  for  RANGER,  SP,  and  varying  values  of 
the  parameter  c  with  larger  values  indicating  more  costly  communication.  RANGER  finds 
paths  two  orders  of  magnitude  greater  than  LP  2  because  it  optimizes  only  for  spreading 
the  flow  as  evenly  as  possible.  Because  it  is  insensitive  to  the  network  performance,  it  will 
perform  arbitrarily  bad  as  c  increases.  In  contrast,  LP  2  tends  to  find  short  paths  when 
communication  is  costly,  and  when  communication  is  very  costly  (c  =  1  means  a  single 
hop  is  as  harmful  as  an  adversary’s  attack),  LP  2  converges  to  the  shortest  paths  strategy. 
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Figure  4.3:  Length  of  solution  pathways  for  RANGER,  shortest  paths  (SP),  and  LP  2  with 
varying  communication  cost. 
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Chapter  5 

Flow  Allocation  in  Adversarial 
Environments:  Non-Zero-Sum  Games 


In  many  domains,  playing  a  strategy  incurs  a  cost  to  the  player  that  depends  only  on  the 
strategy  he  played.  For  example,  in  an  urban  robot  scenario,  robots  must  expend  limited 
energy  or  fuel  in  moving  along  paths,  while  the  cost  of  deploying  a  camera  for  surveillance 
imposes  an  attack  cost  on  the  adversary.  In  this  chapter  we  extend  the  game  to  consider 
these  kinds  of  costs,  using  these  paths  costs  and  attack  costs  as  concrete  examples.  We 
prove  that  these  kinds  of  non-zero-sum  games  are  similar  to  zero-sum  games  where  these 
costs  are  factored  into  both  players’  payoffs,  allowing  us  to  extend  our  linear  programming 
algorithm  to  finding  equilibria  in  the  non-zero  sum  games  as  well.  We  also  consider  several 
further  variations  on  the  basic  game  with  multiple  sinks  and  groups  of  source  nodes. 

A  summary  of  the  notation  used  in  this  chapter  is  provided  in  Table  5.1.  Portions  of  the 
work  in  this  chapter  was  originally  published  in  Okamoto,  et  al.  [46]. 


5.1  Network  Flow  Game  with  Costs  (NFG) 


We  start  with  the  network  flow  game  with  costs.  This  is  identical  to  the  zero-sum  network 
flow  game  described  in  the  previous  section,  but  each  player  now  incurs  a  cost  that  depends 
only  on  his  own  strategy,  which  is  subtracted  from  his  payoff.  Thus  we  have  the  following 
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C 'adversary  (?) 
C’sender(/) 

C 


E 

f 

k 

G  =  (V,E ) 

m 

M 

n 

q 

s 

&  =  {SU...,SK} 

T 

t 

V 

we 


source  requirement  of  s  G  S' 
cost  to  the  adversary  of  playing  q 
cost  to  the  sender  for  playing  / 

attack  cost  vector  for  the  adversary,  with  ca  being  the  cost  of  play¬ 
ing  attack  a  <G  A 
set  of  edges 

sender  strategy,  a  flow  with  f*v  being  the  amount  of  flow  on  edge 
(u,  v)  G  E  for  source  node  s  G  S 

number  of  attacks  that  the  adversary  can  play  simultaneously 
network  with  nodes  V  and  edges  E 
number  of  edges 

harm  matrix  with  Mt)  the  harm  from  sending  1  unit  of  flow  on 
ej  G  E  when  the  adversary  plays  a,  G  A 
number  of  nodes 

an  adversary  mixed  strategy,  often  expressed  as  a  vector  of 
marginal  probabilities  over  A 
set  of  source  nodes 

set  of  groups  of  source  nodes  that  partitions  the  source  nodes 
set  of  sink  nodes 
a  sink  node 
set  of  nodes 

environmental  per-unit-flow  cost  to  sender  of  using  edge  e  G  E 


Table  5.1:  Summary  of  notation  for  Chapter  5 
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payoff  functions: 


^sender (fi  ?)  qM  f  Csender (/) 

(^adversary  (  ft  ?)  qM  f  ^adversary  (?) 


where  C'sender(-)  and  6'adversary(-j  are  the  cost  functions  for  the  sender  and  adversary,  respec¬ 
tively. 

As  concrete  examples  of  these  cost  functions,  we  consider  path  costs  for  the  sender  and  at¬ 
tack  costs  for  the  adversary.  The  path  costs  represent  things  such  as  expenditure  of  energy 
for  robot  movement  or  communication  in  a  multi-hop  sensor  network.  These  costs  reduce 
the  payoff  for  the  sender  but  do  not  affect  the  payoff  to  the  adversary.  Note  that  this  differs 
from  the  costly  transmission  harm  matrix  of  Section  4.2,  which  is  applicable  when  the 
adversary  does  explicitly  care  about  the  path  costs  of  the  sender.  Path  costs  are  computed 
in  the  usual  way,  by  associating  each  edge  e  £  f  with  a  weight  we  representing  the  cost 
of  using  that  edge,  and  summing  the  weights  of  edges  in  each  path.  By  representing  the 
weights  asalxm  vector,  we  can  express  the  sender  cost  function  as 

Csender(/)  =  wf. 

The  adversary  suffers  attack  costs  that  represent  things  such  as  the  monetary  cost  of  de¬ 
ploying  cameras  to  conduct  surveillance  in  an  area  or  the  cost  in  resources  to  conduct  an 
ambush.  These  costs  reduce  the  payoff  of  the  adversary  and  are  incurred  irrespective  of 
whether  the  attack  causes  the  sender  any  harm;  this  may  cause  the  adversary  not  to  utilize 
some  attacks  if  the  costs  exceed  the  benefits.  We  represent  the  costs  by  a  |A|  x  1  vector  c 
where  ca  is  the  cost  of  the  adversary  executing  attack  a.  If  we  assume  the  costs  are  additive 
for  multiple  attacks  (as  in  the  case  of  deployment  costs  of  cameras,  for  instance),  we  can 
represent  the  adversary’s  cost  function  as 


^adversary  (?)  —  ?C. 


To  illustrate  the  importance  of  the  attack  costs  on  the  Nash  equilibrium,  consider  the  net¬ 
work  in  Figure  5.1.  Assume  that  bs  =  1  and  k  =  1  and  that  the  adversary  can  choose  to 
attack  the  top  path  or  the  bottom  path  with  harm  matrix 


M 


102  0  0  0 
0  3  0  0 


Let  fi  denote  the  amount  of  flow  on  edge  er  and  note  that  the  sender’s  strategy  is  fully 
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Figure  5.1:  An  example  of  a  network  with  two  possible  paths. 

specified  by  f\  as  f2  —  l  —  f\.  Let  qx  and  q2  denote  the  probability  of  attacking  the  top 
and  bottom  paths  respectively. 

When  attack  costs  are  zero,  the  adversary  never  has  incentive  not  to  attack,  so  q\  +  q2  =  1. 
In  equilibrium  the  adversary  is  indifferent  between  the  two  attacks  so  102/i  =  3(1  —  f\) 
and  so  f\  =  3/105.  Similarly,  the  sender  is  indifferent  between  the  two  paths  so  102gi  = 
3(1  —  q\)  and  so  qi  =  3/105.  An  intuitive  interpretation  is  that  the  sender  sends  most  of 
his  flow  on  the  bottom  path  because  of  the  lower  potential  for  harm,  while  the  adversary, 
being  able  to  deduce  this,  attacks  the  bottom  path  with  high  probability  because  that’s 
where  most  of  the  flow  is. 

Now  suppose  that  attacking  the  top  path  has  a  cost  Ci  =  100.  The  adversary’s  equilibrium 
strategy  remains  the  same  because  the  sender’s  payoff  hasn’t  changed,  but  now  for  the 
adversary  to  be  indifferent  it  must  be  that  102/i  —  100  =  3f2  so  that  /i  =  103/105.  An 
intuitive  explanation  is  that  the  adversary  attacks  the  top  path  with  low  probability  because 
of  the  high  attack  cost,  and  the  sender,  deducing  this,  sends  most  of  the  flow  on  the  top 
path  despite  the  high  potential  for  harm  because  the  adversary  is  unlikely  to  attack  there. 

The  attack  costs  can  also  cause  the  adversary  to  not  execute  his  maximum  number  of 
attacks  because  the  cost  outweighs  the  harm.  For  example,  if  c \  >  102  then  the  sender  can 
set  /i  =  1  and  a  best  response  by  the  adversary  is  to  choose  q\  =  q2  =  0. 


5.2  Computing  Nash  Equilibrium 

Finding  Nash  equilibria  in  general  non-zero-sum  games  is  computationally  more  expen¬ 
sive  than  finding  equilibria  in  zero-sum  games.  In  addition,  there  may  be  multiple  equi¬ 
libria  with  different  payoffs  for  both  players,  which  can  complicate  the  matter  of  choosing 
a  strategy.  In  this  section  we  show  that  these  concerns  do  not  arise  in  the  network  flow 
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game  with  strategy  costs,  because  the  Nash  equilibria  in  this  game  correspond  to  those  of 
the  zero-sum  game  where  both  payoffs  are  affected  by  costs. 

To  prove  this  we  will  make  use  of  the  following  function  that  includes  the  harm  and  costs 
to  both  players: 


U(f,q)  —  qMf  +  Cinder 


c, 


adversary 


(5.1) 


and  the  following  lemma: 

Lemma  2.  Let  q  be  an  adversary’s  strategy  and  let  f  be  a  sender’s  strategy.  Then 

1.  f  is  a  best  response  to  q  if  and  only  if  f  minimizes  U  (/,  q). 

2.  q  is  a  best  response  to  f  if  and  only  if  q  maximizes  U  (/,  q). 


Proof  We  start  with  claim  1.  By  definition,  /  is  a  best  response  to  q  if  and  only  if  / 
maximizes  Usender(f,  q).  This  happens  if  and  only  if  /  minimizes  —  (/sender (/,  q)  +  oi  for 
any  a  that  is  constant  (with  respect  to  /).  In  particular,  /  is  a  best  response  to  q  if  and  only 

if  it  minimizes  (^sender  {fi  q)  Cadversary(?)  QM  f  T  Csender(/)  ^adversary  (?)  U(f,q). 

The  proof  of  claim  2  is  similar.  □ 


We  can  now  prove  the  theorem: 

Theorem  2.  (/,  q)  is  a  Nash  equilibrium  for  the  network  flow  game  with  strategy  costs  if 
and  only  if  f  minimizes  max,/  U (/,  q')  and  q  maximizes  min//  U (/',  q). 


Proof  We  start  with  the  backward  direction.  Assume  that  /  minimizes  max/  U (/,  <() 
and  q  maximizes  min//  U (/',  q).  We  wish  to  show  that  (/,  q)  is  a  Nash  equilibrium  for  the 
network  flow  game  with  strategy  costs.  To  see  this,  observe  that  /  is  a  minimax  strategy 
in  a  zero-sum  game  where  the  payoff  is  determined  by  U(-),  and  q  is  a  maximin  strategy 
in  the  same  game.  Thus,  by  the  Minimax  Theorem,  /  maximizes  U(f,  q)  and  q  minimizes 
U (/,  q).  Thus  by  Lemma  2,  /  and  q  are  mutual  best  responses  in  the  network  flow  game 
with  strategy  costs,  and  hence  (/,  q)  is  a  Nash  equilibrium  for  the  network  flow  game  with 
strategy  costs. 

We  proceed  next  to  the  forward  direction.  Suppose  that  (/,  q)  is  a  Nash  equilibrium  for 
the  network  flow  game  with  strategy  costs.  We  prove  that  /  must  minimize  the  maximum 
U (/,  <{)  by  contradiction.  Suppose  that  /  is  not  a  minimax  strategy  and  let  /'  be  a  minimax 
strategy.  Then  there  exists  a  marginal  probability  vector  q"  such  that  for  all  marginal 
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probability  vectors  (/,  U (/',  <()  <  U(f.  <[').  In  particular,  for  q'  =  q,  we  get 


U(f,  q)  =  qMf  +  C'sender(//)  -  Cadveisaiy(q) 

<  q  M  f  Csender(/)  C*adversary (?  ) 

<  qM f  +  Csender(/)  —  ^adversary (?)  because  q  is  a  best  response  to  / 

=  U(f,q ) 

But  qMf  T  Csender(/  )  ^adversary  (?)  ^  qM f  +  Csender (/)  ^adversary  (?)  implies  that 
C4e  nder(/',  ?)  =  ?M/'  +  C'sender(/')  <  qM  f  +  C'sender(/)  =  ^sender  (/,  ?),  Which  means  that 

/  is  not  a  best  response  to  g  for  the  sender,  contradicting  (/,  q)  being  a  Nash  equilibrium 
for  the  network  flow  game  with  strategy  costs.  Hence  /  must  minimize  max,,/  U(f.  q'). 

Assume  now  for  the  purposes  of  contradiction  that  q  is  not  a  maximin  strategy.  Let  u  = 
min//  U(f',q )  and  f"  G  argmin//  U(f',q).  Let  ({  be  a  maximin  strategy,  that  is,  <{  G 
argmaXq//  min//  U(f',  q"),  and  let  v!  =  min//  U(f',  q').  Because  q1  is  maximin  and  q  is 
not,  it  follows  that  u'  >  u.  Note  that  by  definition  U (/',  q')  >  u 1  and  U (/',  ?)  >  u  for  all 
f,  and  in  particular  for  f  —  f  we  get  that  U (/,  ?')  >  v!  >  u  =  U (/",  q).  Thus, 

U(f,q')=q'Mf  +  ^sender  (/*)  ^adversary  (?) 

>  qMf"  + 

^sender)./  )  C*adversary  (?) 

>  ?M/  +  Csender(/)  —  ^adversary (?)  because  f  is  a  best  response  to  q 

=  U(f,q ) 

But  this  implies  that  ?,M/-C,adversary(g/)  >  gM/-Cadversary(g),  that  is,  that  f/adversary(/,  </)  > 
Adversary  (/,  ?)•  This  contradicts  the  assumption  that  g  is  a  best  response  to  /  in  the  network 
flow  game  with  strategy  costs.  Thus  q  must  be  a  maximin  strategy.  □ 


Because  of  Theorem  2,  finding  an  equilibrium  sender  strategy  reduces  to  finding  a  mini- 
max  strategy.  We  can  extend  the  network  flow  LP  we  used  in  the  previous  section  to  get 
the  linear  program  LP  4.  This  LP  differs  from  LP  2  in  that  the  costs  are  explicitly  factored 
in  to  objective  function.  The  attack  costs  are  subtracted  from  the  harm  that  would  be  in¬ 
flicted  by  an  attack  in  Eq.  5.3,  reflecting  the  fact  that  the  adversary’s  payoff  is  reduced  by 
the  attack  cost.  The  effect  of  edge  costs  on  the  sender  is  directly  represented  by  adding  the 
edge  costs  to  the  objective  function  that  is  being  minimized. 
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LP  4  Equilibrium  Sender  Strategy  with  Costs 

Input:  G,  b,  t,  M,  c,  k 

Output:  /,  H,  X 


Minimize  kH  +  >  Xa  +  wf 

f,H,  A 

a&A 

(5.2) 

subject  to: 

H  >  rOW a[M]f  -  Ca-Xa 

Va  e  A 

(5.3) 

by  T  ^  ^  fuv  ^  ^  fvu 

Wv  e  V  \  {t} 

(5.4) 

(u,v)eE  (v,u)£E 

/>  0 

(5.5) 

A  >  0 

(5.6) 

5.2.1  Computing  the  Adversary’s  Strategy 

We  can  compute  an  equilibrium  strategy  for  the  adversary  using  LP  5,  which  is  the  dual  of 
LP  4.  The  q  variables  represent  the  adversary’s  mixed  strategy  in  the  usual  way.  The  vari¬ 
able  rv  represents  the  minimum  per-unit-flow  harm  plus  edge  costs  that  the  sender  would 
suffer  if  he  played  optimally  against  q  while  starting  from  v  G  V.  The  main  constraint 
is  in  Equation  (5.8):  for  an  edge  (u,  v ),  the  sender  can  do  no  worse  from  u  than  he  does 
from  v  plus  the  harm  and  edge  costs  of  sending  flow  across  (u,  v).  The  objective  in  Equa¬ 
tion  (5.7)  is  to  maximize  the  total  harm  and  costs  (weighted  by  the  amount  of  flow)  from 
each  source  node,  less  the  attack  costs.  By  Theorem  2  this  finds  an  equilibrium  strategy 
for  the  adversary  in  the  NFG. 

However,  it  only  finds  an  equilibrium  strategy.  What  if  we  want  to  find  an  equilibrium 
strategy  for  the  that  gives  the  sender  the  worst  payoff? 

Let  a  solution  to  LP  5  be  (q*,  r*)  with  objective  function  value  A"*.  We  know  that  any  other 
solution  must  also  have  objective  function  value  A*,  but  this  may  result  in  different  payoffs 
to  the  sender  in  the  non-zero-sum  NFG.  We  want  to  find  an  equilibrium  strategy  for  the 
adversary  that  yields  the  worst  payoff  to  the  sender  in  NFG.  This  worst  case  equilibrium 
strategy  depends  on  the  sender’s  strategy  /,  which  we  previously  computed  with  LP  4. 

We  use  LP  6,  which  maximizes  the  harm  to  the  sender  while  ensuring  that  all  constraints 
of  LP  5  are  satisfied  and  that  the  solution  would  also  be  optimal  for  LP  5. 

Let  a  solution  to  LP  6  be  qworst ;  rworst_  Things  to  note: 


77 


LP  5  Equilibrium  Adversary  Strategy  with  Costs 

Input:  G,  b ,  t,  M,  c,  w,  k 

Output:  q,  r 


Maximize 

bsr° 

)  -  Y Caqa 

(5.7) 

q,r 

\s£S  , 

'  aeA 

subject  to: 

ru  <  gTcol( 

[u,v)[M]  + 

V v  u,v ) 

V(w,  v)  G  E 

(5.8) 

rt  =  0 

(5.9) 

qa  <  1 

Mae  A 

(5.10) 

Y <la<k 

(5.11) 

aeA 

q,  r  >  0 

(5.12) 

LP  6  Worst  Case  Equilibrium  Adversary  Strategy  with  Costs 

Input:  G,  b,  t,  M,  c,  w,  k,  X*,  f 

Output:  q,  r 


Maximize  qM f 

q,r 

subject  to: 


-  qTc ol(u>„)  [M]  +  ru  -  rv  <  V(w,  v)  G  E 

rt  =  0 

qa  <  1  Va  G  A 

a&A 

q,  r  >  0 


(5.13) 


(5.14) 

(5.15) 

(5.16) 

(5.17) 

(5.18) 

(5.19) 
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•  qworst ;  r™orst  is  aiso  a  solution  to  LP  5.  (Any  feasible  solution  to  LP  6  is  a  feasible 
solution  to  LP  5,  and  by  Eq  (5.14),  qworst ^  rworst  maximizes  the  objective  function 
of  LP  5.) 

•  LP  6  always  has  a  solution.  (LP  5  always  has  a  solution,  so  LP  6  must  have  a 
solution.) 

•  Although  the  sender’s  payoff  in  the  NFG  includes  transmission  costs,  this  is  not 
included  in  the  objective  function  of  LP  6  as  the  term  is  wf  which  is  constant  in  this 
program.  The  weights  w  do  appear  in  the  constraints. 

Let  EQadv  be  the  set  of  equilibrium  adversary  strategies.  Let  W  EQadv(f)  be  the  set  of 
worst  case  equilibrium  adversary  strategies  for  an  equilibrium  sender  strategy  /.  That  is 

WEQadv(f)  =  {qeEQadv:qMf=  max  q'Mf}  (5.20) 

q'eEQadv 


Note  that  WEQadv(f )  is  non-empty. 

Theorem  3.  LP  6  computes  a  worst  case  equilibrium  adversary  response  to  f. 

We  can  show  that  this  is  also  a  worst  case  for  all  equilibrium  sender  strategies,  not  just  the 
particular  choice  of  /.  Let  EQsend  be  the  set  of  equilibrium  sender  strategies.  We  start  by 
noting  the  interchangeability  of  equilibria  in  the  zero-sum  game  related  to  NFG: 

Lemma  3.  Let  f  G  EQsend  and  q  G  EQadv.  Then  (/,  q)  is  an  equilibrium  strategy  for  the 
zero-sum  game  with  payoff  function  U  (/,  q)  =  qMf  +  Csender  —  C adversary  for  the  adversary 
and  —U (/,  q)  for  the  sender. 

Proof  From  the  minimax  theorem.  □ 

We  then  use  this  to  show  the  interchangeability  of  equilibria  in  NFG: 

Lemma  4  (Interchangeability  of  equilibria  of  NFG).  Let  f  G  EQsend  and  q  G  EQadv. 
Then  (/,  q)  is  an  equilibrium  strategy  for  the  non-zero-sum  NFG. 

Proof.  From  Femma  3  and  Theorem  2.  □ 

We  are  now  ready  to  prove  that  the  adversary’s  worst  case  equilibrium  strategies  for  NFG 
do  not  depend  on  the  specific  equilibrium  strategy  played  by  the  sender. 

Theorem  4.  Let  /,  f  G  EQsend.  Then  WEQadv(f )  =  WEQadv(f). 

Proof.  We  prove  this  by  contradiction.  Fet  q  G  W EQadv(f)  and  <(  G  W EQadv(f). 
Assume  that  q  f  WEQadv(f). 
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By  interchangeability,  (/,  q),  (/',  q ),  (/',  q'),  and  (/,  g')  are  all  equilibria.  Thus,  Usender(f,  q) 
U sender (/'/ 5  g)  and  f/sender(/,  <l  )  =  lender (/',  g')  so  that  the  sender  is  indifferent  between  / 
and  f  when  the  adversary  plays  q  or  q'.  Thus  we  get  the  following  system: 


I  —q'Mf  -  C'enderC/')  =  ~q’ M  f  -  Cinder (/) 

-  jCseoderl/')  =  ~qM f  -  Csendel(f ) 
-  q’Mf  +  qMf  =  - q'Mf  +  qMf 


(5.21) 

(5.22) 


Because  g  ^  WEQadv(f),  it  must  be  that  qMf  <  q'M f .  Thus  with  Eq  (5.22)  we  get 

-  q’Mf  +  qMf  =  -q’Mf  +  qMf  <  -q'Mf  +  q'Mf  =  0  (5.23) 

==►  qMf  <  q'Mf  (5.24) 


Because  q  G  EQadv(f),  it  follows  that  q'M f  <  qM /,  and  thus  with  Eq  (5.24)  we  get  the 
contradiction  that  qM  f  <  qM  f. 

Therefore  q  e  WEQadv(f )  and  so  WEQadv(f )  C  WEQadv(f).  By  symmetry  it  follows 
that  WEQadv(f)  C  WEQadv(f)  and  so  WEQadv{f )  =  WEQadv(f).  □ 


By  Theorem  4  and  Theorem  3,  we  have  shown  that  LP  6  finds  an  equilibrium  strategy  for 
the  adversary  that  yields  the  worst  payoff  for  the  sender  in  NFG. 


5.3  NFG  Example  Problem 

In  this  section  we  work  through  an  example  of  modeling  a  real  world  problem  as  a  net¬ 
work  flow  game  with  costs.  In  this  example  we  consider  the  problem  of  police  officers  in 
Kabul,  Afghanistan  responding  to  an  incident  at  the  Hotel  Inter-Continental  Kabul,  one  of 
Kabul’s  best  known  hotels  and  among  the  most  frequently  visited  by  foreigners,  especially 
westerners.  On  June  28,  2011,  it  was  the  target  of  a  6-hour-long  attack  by  armed  terrorists 
that  left  at  least  21  dead,  including  all  9  attackers,  after  a  siege  by  Afghanistan  police  and 
NATO  forces  [55].  In  a  future  incident,  terrorists  may  try  to  delay  the  arrival  of  security 
forces  by  obstructing  roads  leading  to  the  hotel.  These  roadblocks  must  be  placed  in  a 
limited  amount  of  time  as  they  must  be  set  up  before  the  police  use  the  roads,  but  erecting 
them  too  early  may  tip  off  the  security  forces  to  the  impending  attack  on  the  hotel. 

We  model  this  situation  as  a  network  flow  game  with  the  sender  representing  the  police 
and  the  adversary  representing  the  terrorists  trying  to  delay  the  arrival  of  the  police  at  the 
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hotel.  The  sender  chooses  paths  through  city  streets  for  the  police  to  take  from  their  police 
stations  to  the  hotel.  The  adversary  chooses  roads  for  the  terrorists  to  obstruct  to  delay 
the  police.  Thus  the  adversary  “attacks”  edges  (corresponding  to  roadblocks  on  individual 
roads),  and  harm  is  calculated  using  a  form  of  path  intersection  harm  matrix  where  the 
sender  incurs  harm  if  and  only  if  the  police  use  a  road  that  has  been  obstructed  by  the 
terrorists.  We  assume  that  the  terrorists  start  from  a  known  location,  thus  restricting  which 
roads  they  can  obstruct  in  the  amount  of  time  they  have  available. 

Figure  5.2  shows  the  graph  of  the  roads  in  a  roughly  6.5  km  by  6.5  km  section  of  west¬ 
ern  Kabul,  constructed  from  GIS  data  from  the  Metro  Extracts  1  of  the  OpenStreetMap 
database.  This  graph  has  3655  nodes  and  8704  edges.  The  square  near  the  center  of  the 
graph  depicts  the  location  of  the  Hotel  Inter-Continental  Kabul  and  is  the  sink  node  for  the 
network  flow  game.  The  upper  and  lower  squares  depict  the  locations  of  the  two  nearest 
police  stations  and  are  the  source  nodes  for  the  network  flow  game.  The  last  square,  on  the 
right,  is  on  a  major  road  running  through  the  area  and  is  the  entry  point  into  the  region  for 
the  terrorists.  We  assume  that  the  terrorists  have  time  to  travel  up  to  5  km  along  city  streets 
to  place  obstructions,  and  that  they  do  not  place  roadblocks  within  1  km  of  the  hotel  or  the 
police  stations,  as  these  are  the  most  securely  patrolled  locations.  This  limits  the  adversary 
to  choosing  from  5324  edges  to  attack,  shown  in  bold  in  Figure  5.2.  Attack  costs  to  rep¬ 
resent  the  risk  of  detection  and  the  use  of  men  and  materials  in  placing  roadblocks,  with 
the  attack  costs  proportional  to  the  distance  the  terrorists  must  travel  to  reach  the  attacked 
edge  from  their  starting  location. 

We  consider  three  ways  of  representing  and  solving  this  problem  as  a  network  flow  game, 
using  different  models  for  the  adversary.  The  first,  the  NFG  approach,  directly  uses  the 
formulation  developed  in  Section  5.2,  with  the  adversary’s  pure  strategies  being  any  subset 
of  up  to  k  attacks  on  edges  within  range,  and  seeking  to  maximize  his  expected  payoff  of 
the  harm  minus  the  attack  costs.  In  the  second  approach,  the  budgeted  NFG  approach, 
the  attack  costs  do  not  factor  into  the  adversary’s  payoff,  but  he  is  instead  provided  with  a 
budget  that  he  must  meet  in  expectation.  The  third  approach,  the  normal-form  approach, 
also  uses  a  budget  but  the  adversary  must  satisfy  the  stronger  condition  of  meeting  the 
budget  with  every  pure  strategy,  where  the  attacked  edges  in  a  pure  strategy  must  lie  along 
a  single  path.  We  describe  these  approaches  in  more  detail  next. 

NFG  approach.  This  approach  directly  uses  the  approach  from  Section  5.2,  calculating 
the  sender’s  equilibrium  strategy  using  FP  4  and  the  adversary’s  equilibrium  strategy  using 
FP  5.  The  adversary’s  choice  of  strategies  is  constrained  by  the  number  of  edges  k  that 
he  can  simultaneously  attack  and  by  the  costs  of  the  attacks,  which  are  deducted  from  his 
payoff.  These  reflect  a  situation  where  there  are  k  terrorists  at  the  starting  location,  each 

1  http://metro.teczno.com/ 
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LP  7  Budgeted  NFG:  Equilibrium  Adversary  Strategy  with  Budgeted  Costs 

Input:  G ,  b,  t,  M,  c,  w,  k,  X*,  f 

Output:  q,  r 


Maximize  Y"'  bsrs 

(5.25) 

seS 

subject  to: 

ru  >  qTcol (U,V)[M]  +rv  + 

V(w,  v)  E  E 

(5.26) 

rt  =  0 

(5.27) 

qa<  l 

Mae  A 

(5.28) 

Va<k 

(5.29) 

aeA 

caqa  <  C 

(5.30) 

a&A 

q,  r  >  0 

(5.31) 

one  capable  of  deploying  a  single  road  block,  and  the  terrorists  collectively  choose  roads 
to  obstruct  on  order  by  balancing  the  expected  travel  time  of  the  police  with  the  costs  of 
erecting  road  blocks. 

Budgeted  NFG  approach.  In  this  approach,  the  attack  costs  do  not  directly  factor  into 
the  adversary’s  payoff.  Instead,  the  adversary  has  a  fixed  budget  C  which  he  uses  to  cover 
attack  costs.  We  solve  this  problem  using  a  modified  version  of  the  NFG  approach.  The 
adversary’s  equilibrium  strategy  is  computed  using  LP  7.  This  is  identical  to  LP  5  except 
that  the  expected  attack  costs  have  been  removed  from  the  objective  (Equation  (5.25)),  and 
there  is  a  new  constraint  requiring  the  expected  attack  costs  to  be  no  more  than  the  budget 
(Equation  (5.30)).  Note  that  this  only  guarantees  that  the  budget  is  met  in  expectation, 
which  means  that  some  pure  strategies  selected  from  a  mixed  strategy  may  exceed  the 
budget,  even  if  the  mixed  strategy  itself  meets  the  budget  constraint. 

The  sender’s  equilibrium  strategy  is  computed  using  the  dual  of  LP  7,  shown  in  LP  8.  This 
LP  has  a  new  variable  X  that  corresponds  to  the  budget  constraint  of  Equation  5.30  in  the 
adversary’s  LP. 

Normal-Form  approach.  In  this  approach  a  single  terrorist  vehicle  begins  at  the  starting 
location  and  moves  through  the  road  network,  placing  up  to  k  road  blocks  in  the  desired 
locations.  The  attack  costs,  which  are  proportional  to  the  distance  the  terrorists  must 
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LP  8  Budgeted  NFG:  Equilibrium  Sender  Strategy  with  Budgeted  Costs 

Input:  G,  b ,  t,  M,  c,  k 

Output:  /,  II .  X 


Minimize  kH  +  V"  Xa  +  w  f  +  XC 

a&A 

(5.32) 

subject  to: 

H  >  row a[M\f  -Xa-  caX 

Va  G  A 

(5.33) 

bv  T  ^  ^  fuv  ^  ^  fvu 

Vu  e  V  \  {t} 

(5.34) 

( u,v)£E  ( v,u)£E 

/>  o 

(5.35) 

A  >  0 

(5.36) 

X  >  0 

(5.37) 

travel,  depends  on  the  specific  path  taken  by  the  terrorists  to  reach  the  blockaded  roads. 
In  addition,  we  require  that  the  budget  constraint  is  met  by  every  pure  strategy,  not  just  in 
expectation  for  mixed  strategies. 

As  a  result  of  these  assumptions,  not  all  subsets  of  A  of  size  at  most  k  are  feasible  pure 
strategies  for  the  adversary.  For  example,  two  edges  may  be  individually  within  range  of 
the  terrorists’  starting  location,  but  are  too  far  for  the  terrorists  to  blockade  both  of  them 
while  staying  under  budget.  These  constraints  cannot  be  represented  compactly  using  the 
NFG  approach  of  relying  solely  on  the  marginal  probabilities  qa  of  each  attack  a  e  /l. 
Instead  we  use  the  more  general  but  less  efficient  normal  form  representation  by  explicitly 
enumerating  the  subsets  of  A  of  size  at  most  k,  checking  which  ones  meet  the  budget 
constraint  to  determine  the  feasible  pure  strategies,  then  explicitly  computing  the  payoffs. 
We  then  solve  this  normal  form  game  using  a  standard  minimax  LP  approach. 

This  approach  uses  a  strategy  space  for  the  adversary  that  is  exponentially  larger  than 
the  strategy  space  used  by  the  other  two  approaches.  Because  of  this,  the  normal-form 
approach  scales  poorly  with  k. 


5.3.1  Comparison  of  Approaches 

We  first  compare  the  three  approaches  for  k  —  2.  Figure  5.3  shows  the  equilibrium  solution 
for  the  NFG  approach.  The  sender’s  flows  are  shown  using  dark  blue  lines  (near-black  lines 
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Figure  5.3:  Equilibrium  solution  of  NFG  approach  with  k  —  2. 
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in  black-and-white),  with  thicker  lines  indicating  a  greater  amount  of  flow.  The  adversary’s 
attacks  are  shown  using  red  lines  (gray  lines  in  black-and-white)  on  attacked  edges,  with 
thicker  lines  indicating  a  higher  marginal  probability  of  attack.  Most  of  the  adversary’s 
attacks  involve  edges  from  the  northern  police  station.  This  is  because  they  are  closer 
to  the  adversary’s  starting  location  and  hence  cheaper  for  the  adversary,  and  in  the  NFG 
approach  the  attack  costs  are  deducted  from  the  adversary’s  payoff.  The  adversary  does 
attack  an  edge  on  a  path  from  the  southern  police  station,  but  only  with  low  probability. 

There  is  also  relatively  little  branching  of  the  sender’s  flows  because  any  reduction  in  harm 
from  the  adversary’s  attacks  would  not  compensate  for  the  increase  in  travel  time  from 
taking  longer  paths.  For  example,  police  from  the  northern  police  station  could  head  west 
before  turning  south  to  approach  the  hotel,  joining  with  the  flow  from  the  southern  police 
station  and  approach  the  hotel  from  the  west.  This  would  greatly  reduce  the  probability  of 
encountering  a  terrorist  roadblock,  but  would  also  greatly  increase  the  travel  time  due  to 
the  longer  routes. 

Note  also  that  the  edges  attacked  by  the  adversary  form  a  cut  in  the  flow  from  the  northern 
police  station  but  not  for  the  southern  police  station.  Because  in  equilibrium  the  sender 
is  indifferent  between  the  paths  he  has  chosen  to  send  flow  on,  this  means  that  the  effect 
of  the  southern  attack  is  to  decrease  the  value  of  the  shorter  path  with  the  attacked  edge, 
to  make  it  equally  valuable  as  the  longer  path  that  is  not  attacked.  All  paths  with  flow 
from  the  northern  police  station  include  at  least  one  edge  that  is  attacked  by  the  adversary 
with  non- zero  probability,  and  the  marginal  probabilities  of  attack  are  balanced  to  make 
the  sender  indifferent. 

Figure  5.4  shows  the  equilibrium  solution  for  the  budgeted  NFG  approach.  The  main 
difference  from  the  NFG  approach  is  that  the  adversary  is  much  more  likely  to  attack 
edges  near  the  southern  police  station.  This  is  because  the  attack  costs  do  not  factor  into 
the  adversary’s  payoff.  Thus  a  high  cost  of  attack  will  not  dissuade  the  adversary  from 
playing  that  attack  if  there  remaining  budget  left  and  no  better  alternative. 

Figure  5.5  shows  the  equilibrium  solution  for  the  normal- form  approach.  The  result  for  the 
adversary  is  similar  to  the  budgeted  NFG  approach.  Like  the  budgeted  NFG  approach  and 
unlike  the  NFG  approach,  the  attack  costs  do  not  directly  factor  into  the  adversary’s  payoff, 
possibly  enabling  the  adversary  to  attack  edges  near  the  southern  police  station.  However, 
the  strict  budget  constraint  requires  that  both  attacks  on  edges  in  a  pure  adversary  strategy 
must  be  reachable  from  the  start  node  in  the  same  path.  Thus,  when  the  adversary  travels 
to  attack  near  the  southern  police  station,  it  chooses  to  blockade  two  edges. 

We  next  compare  the  NFG  and  budgeted  NFG  approaches  for  k  =  10.  Because  of  its 
poor  scalability,  the  normal-form  approach  was  not  able  to  be  used.  Figure  5.6  shows 
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Figure  5.4:  Equilibrium  solution  of  budgeted  NFG  approach  with  k  —  2. 
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Figure  5.5:  Equilibrium  solution  of  normal-form  approach  with  k  =  2. 
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Figure  5.6:  Equilibrium  solution  of  (a)  NFG  approach  and  (b)  budgeted  NFG  approach 
with  k  =  10. 


the  equilibrium  solution  for  the  NFG  and  budgeted  NFG  approaches.  Compared  to  the 
solutions  for  k  =  2,  the  sender’s  flows  are  much  more  extensively  branched.  This  is 
because  the  adversary  is  able  to  attack  five  times  as  many  edges  as  in  the  k  =  2  setting, 
providing  a  stronger  incentive  for  the  sender  to  reduce  the  amount  of  flow  on  any  single 
edge.  Note  that  as  in  the  k  =  2  case,  the  adversary  under  the  budgeted  NFG  approach 
is  much  more  likely  to  attack  edges  near  the  southern  police  station  than  he  is  under  the 
NFG  approach.  The  corollary  of  this  is  that  the  adversary  is  more  likely  to  attack  near  the 
northern  police  station  for  the  NFG  approach  than  for  the  budgeted  NFG  approach.  As  a 
result,  with  the  NFG  approach  the  sender  has  greater  incentive  to  reduce  the  flow  on  any 
single  edge,  leading  to  even  more  branching  in  the  flows  from  the  northern  police  station 
than  seen  in  the  budgeted  NFG  approach. 

These  differences  are  brought  out  even  further  for  k  =  20,  as  shown  in  Figures  5.7.  With 
the  NFG  approach,  the  adversary’s  attacks  in  the  north  provide  the  sender  with  incentive 
to  branch  further,  even  using  the  long  counter-clockwise  path  around  the  hotel  to  approach 
the  hotel  from  the  west.  Similarly,  with  the  budgeted  NFG  approach,  the  adversary’s 
attacks  in  the  south  provide  the  sender  with  incentive  to  use  a  long  path  to  approach  the 
hotel  from  the  east. 
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Figure  5.7:  Equilibrium  solution  of  (a)  NFG  approach  and  (b)  budgeted  NFG  approach 
with  k  =  20. 


5.4  Network  Flow  Game  with  Multiple  Sinks  (NFG-MS) 


In  the  games  that  we  have  considered  so  far  there  is  a  single  known  sink.  This  models 
domains  like  a  robot  team  where  there  is  a  single  known  location  (e.g.,  the  location  of  a 
joint  task)  that  all  robots  must  travel  to,  or  a  sensor  network  where  there  is  a  single  base 
station  capable  of  performing  data  fusion.  However,  in  many  domains  there  are  multiple 
sinks  that  the  source  nodes  must  chose  from:  there  are  multiple  joint  tasks  that  the  robot 
team  can  perform,  or  multiple  base  stations  that  the  sensors  could  transmit  their  data  to. 

The  network  flow  game  with  multiple  sinks  (NFG-MS)  extends  the  network  flow  game 
with  costs  from  a  single  sink  t  e  V\S  to  a  set  of  sinks  T  C  V\S.  For  each  sink  t  G  T  there 
is  a  reward  rt  >  0  that  the  sender  receives  for  using  that  sink.  This  models  heterogeneity 
between  sinks,  such  as  some  tasks  being  more  important  or  some  base  stations  having  more 
battery  power  or  computational  power.  In  the  absence  of  an  adversary  the  optimal  behavior 
for  the  sender  is  to  choose  a  sink  that  maximizes  utility  (the  reward  minus  the  costs). 
However,  this  kind  of  deterministic  behavior  is  vulnerable  to  attack.  Thus,  in  adversarial 
environments  the  agents  will  have  to  balance  reward,  costs,  and  harm  by  randomizing  over 
both  the  sink  to  use  and  the  paths  to  the  sink.  When  the  flows  are  interpreted  as  mixed 
strategies  in  the  path  game,  the  sender  in  each  pure  strategy  must  choose  paths  for  each 
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source  node  to  the  same  sink.  This  is  achieved  by  requiring  that  for  each  sink  t  G  T,  the 
source  nodes  all  send  the  same  fraction  of  their  flow  to  t.  For  convenience  we  assume  that 
the  sources  all  must  transmit  unit  flow,  that  is,  bs  —  1  for  all  s  G  S. 

LP  9  computes  an  equilibrium  sender  strategy  for  the  network  flow  game  with  multiple 
sinks.  In  contrast  to  the  previous  LPs  which  used  a  single  flow  approach,  this  LP  uses 
multicommodity  flows,  with  a  separate  type  of  flow  for  each  source.  The  flow  for  source 
s  6  S'  is  denoted  by  fs.  For  readability  we  use  placeholder  variables  to  represent  the 
various  components  of  the  payoff:  R  for  the  expected  reward,  H  for  the  expected  harm 
(less  attack  costs),  and  G,cdgc  for  the  expected  edge  costs.  For  each  t  G  T.  the  variable  d, 
represents  the  marginal  probability  of  t  being  chosen  as  the  sink.  The  expected  reward  is 
thus  dr  (Eq.  (5.39)).  The  harm  is  calculated  as  before  except  with  the  sum  over  all  types  of 
flows,  as  shown  in  Eqs.  5.40  and  5.41.  The  edge  costs  are  also  calculated  as  before  except 
with  the  sum  over  all  types  of  flows  in  Eq.  5.42.  The  flow  conservation  requirements  have 
been  extended  to  each  individual  type  of  flow,  and  dt  explicitly  models  the  amount  of  flow 
that  is  deposited  in  sink  t.  By  using  a  single  variable  dt  for  all  types  of  flow,  we  ensure 
that  the  the  source  nodes  all  use  t  as  a  sink  for  the  same  fraction  of  their  flow. 

Note  that  in  the  previous  network  flow  LPs,  the  source  nodes  were  required  to  transmit  all 
of  their  flow  to  the  sink.  In  NFG-MS,  the  source  nodes  may  end  up  only  sending  some  of 
their  flow  to  the  sinks.  This  is  reflected  in  Eq.  (5.46),  where  the  sum  of  the  d  variables  is 
required  to  be  less  than  or  equal  to  one,  rather  than  equal  to  one.  This  is  because  in  some 
cases  the  sum  of  the  expected  costs  and  harm  may  exceed  the  expected  reward  for  using  a 
sink. 


5.5  Network  Flow  Game  with  Multiple  Groups  (NFG-MG) 

In  NFG-MS,  all  source  nodes  are  required  to  send  to  the  same  sink  in  a  pure  strategy. 
This  makes  sense  when  the  source  nodes  correspond  to  a  cooperative  group  like  a  team  of 
robots  performing  a  joint  task  or  a  set  of  sensors  tracking  the  same  object.  However,  in 
most  domains  there  are  multiple  groups:  multiple  subteams  performing  different  tasks  or 
different  sensors  tracking  different  objects. 

The  network  flow  game  with  multiple  groups  (NFG-MG)  extends  NFG-MS  to  handle  these 
cases  and  is  solved  (for  the  sender)  by  LP  10.  The  source  nodes  are  partitioned  into  groups 
&  =  (S1!, . . . ,  ,S'|e|}.  The  reward,  edge  costs,  and  harm  matrix  are  indexed  by  group,  r), 
<,  and  Ml  for  all  i  G  (1, . . .  |«S| },  t  G  T,  and  e  G  E.  All  source  nodes  within  a  group 
must  send  flow  to  the  same  sinks  in  the  same  proportions.  Furthermore,  sinks  can  only 
be  used  by  a  single  group  at  a  time;  this  corresponds  to  each  task  only  being  able  to  be 
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LP  9  Equilibrium  Sender  Strategy  with  Multiple  Sinks 

Input:  G,  b,  T,  M,  c,  k 

Output:  f,  H,  X 


Maximize  R  —  H  —  6,edgc 

subject  to: 

(5.38) 

R  =  dr 

(5.39) 

H  =  kH'  +  J2Xa 

aeA 

(5.40) 

H'  >  rowa[M](y]  fs)  -ca-Xa 

seS 

Va  G  A 

(5.41) 

Cedge  = 

s£S 

(5.42) 

k+  Y1  /->  E  /'« 

(u,v)£E  (v,u)gE 

Vu  G  V  \  {t},  Vs  G  S' 

(5.43) 

E  /«><*«+  E  At. 

(u,t)£E  (t,u)£E 

Vt  G  T,  Vs  G  S' 

(5.44) 

d,<  E  fu, 

(u,t)^E 

Vt  G  T,  Vs  G  S' 

(5.45) 

dt = i 

t&T 

(5.46) 

/>  0 

(5.47) 

A  >  0 

(5.48) 

d  >  0 

(5.49) 
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performed  once,  or  each  base  station  only  able  to  perform  fusion  for  a  single  object  at  a 
time.  This  assumption  can  be  relaxed  either  by  modifying  the  constraints  (Eq.  (5.59))  of 
LP  10  or  by  adding  multiple  copies  of  the  sink  nodes  as  desired. 


5.6  Bayesian  Games:  Dealing  with  Uncertainty 

In  many  security  applications,  it  is  unrealistic  to  assume  that  complete  information  on  the 
adversary  is  available  because  adversaries  are  hostile  and  usually  secretive.  In  military  and 
law  enforcement  domains,  intelligence  analysts,  criminologists,  and  other  experts  collect 
relevant  data  on  real  and  possible  adversaries  and  develop  inherently  uncertain  estimates 
of  their  capabilities  and  motives.  In  this  section  we  represent  that  uncertainty  as  proba¬ 
bility  distributions  over  the  maximum  number  of  attacks  that  they  can  execute,  the  harm 
matrices,  and  the  attack  costs.  We  develop  ways  to  reason  strategically  over  this  incom¬ 
plete  information  by  adopting  the  Bayesian  game  framework  and  find  polynomial  time 
algorithms  for  finding  equilibria. 

For  simplicity,  we  present  our  approach  in  the  context  of  NFG,  the  single-group,  single¬ 
sink  network  flow  game  with  costs  solved  by  FP  4  and  FP  5  for  the  case  without  uncer¬ 
tainty.  It  is  straightforward  (although  requiring  more  elaborate  notation)  to  extend  our 
approach  to  NFG-MS  and  NFG-MG. 

5.6.1  Uncertain  k 

In  many  situations  it  is  not  possible  for  the  sender  to  know  the  adversary’s  capabilities  with 
certainty.  The  sender  can  act  as  if  he  has  the  full  knowledge,  but  he  then  might  perform 
badly.  For  example,  suppose  that  the  game  is  the  same  as  in  Figure  5.1,  but  now  k  —  2. 
In  equilibrium,  the  adversary  strategy  is  qi  =  1/34  and  q2  =  1,  and  the  sender  strategy  is 
qi  =  100/102  and  q2  =  2/102.  The  expected  harm  for  the  sender  will  thus  be  3.  However, 
if  the  sender  does  not  know  that  k  =  2  now,  and  continue  to  play  his  strategy  for  k  —  1, 
the  adversary  will  exploit  it  and  will  always  attack  v\  and  v2.  The  sender’s  harm  will 
thus  increase  to  100.116.  Therefore,  when  the  sender  does  not  sure  about  the  exact  value 
of  k,  he  will  have  to  estimate  it.  We  represent  this  by  a  probability  distribution  q  over 
possible  values  of  k ,  which  we  assume  is  known  to  both  players.  Given  this  distribution, 
we  formulate  the  sender’s  problem  as  a  Bayesian  game.  A  Bayesian  game  is  one  in  which 
information  about  characteristics  of  the  other  players  is  incomplete.  There  is  a  probability 
distribution  over  possible  types  for  each  player,  and  the  type  of  a  player  determines  that 
player’s  payoff  function.  In  our  case,  the  sender  has  only  one  type,  and  the  type  of  the 
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LP  10  Equilibrium  Sender  Strategy  with  Multiple  Groups 

Input:  G,b,&,  T,  M,  c,  k 

Output:  f,  H,  A 


Maximize  R  —  H  —  Cedge 

f,R,H,H',C^,d,  A 

subject  to: 

|6 

(5.50) 

*=EE« 

i= 1  teT 

(5.51) 

H  =  kH'  +  ^  Aa 

aeA 

/N  \ 

(5.52) 

#'>  (  ^rowjM^/M  —  ca  —  Aa 

si 

Mae  A 

(5.53) 
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i=l  e£E 

(5.54) 
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V  1  /  y  UV  —  /  y  */  VU 

(u,v)GE  (v,u)eE 
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(5.55) 
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|6| 
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(5.58) 
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MteT 
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LP  11  Equilibrium  Sender  Strategy  in  a  Bayesian  game  (uncertain  k ) 

Input:  G,  M,  A,  c,  k,  Pr (k) 

Output:  f,{Rk},{Xk} 


A\ 

Minimize  w  f  +  >  Pr(fc) 

f,{Rk},{  Afc}  fV 


subject  to: 

Rk  >  row a[M]f  -ca-  Xk 


^  '  fvu  bv  4~  ^  ^  fuv 

(v,u)EE  (u,v)eE 


fuv>  0 

A^>0 

Rk>  0 


(5.63) 

Va  e  A,\/k  G  [1..  A  ] 

(5.64) 

Vv  G  V  \  {t} 

(5.65) 

V(u,v)  G  E 

(5.66) 

Va  G  A,Vk  G  [1..  A  ] 

(5.67) 

Wk  G  [1..  A  ] 

(5.68) 

adversary  is  determined  by  the  value  of  k.  We  denote  the  probability  that  the  adversary  is 
of  type  k  as  Pr(fc). 

The  sender’s  optimal  equilibrium  strategy  can  be  computed  using  LP  11.  This  LP  is  sim¬ 
ilar  to  LP  4,  except  that  instead  of  minimizing  maximum  adversary  expected  payoff  for  a 
specific  value  of  k,  it  minimizes  the  weighted  sum  of  the  expected  rewards  of  every  pos¬ 
sible  k,  weighted  by  their  possibilities.  The  number  of  variables  and  constraints  is  still 
polynomial  in  n  and  |A|  and  so  this  LP  can  be  solved  in  polynomial  time. 

We  must  verify  that  in  the  Bayesian  game,  minimizing  the  adversary’s  maximum  expected 
payoff  also  maximizes  the  sender’s  expected  payoff. 

Lemma  5.  Let  f  be  a  sender  strategy,  {qk}  be  an  adversary  strategy,  and  Pr (k)  be  a 
probability  distribution  over  values  ofk.  Then  f  is  a  best  response  to  {qk}  if  and  only  if  f 
minimizes  expected  reward  relative  to  q  and  Pr (k). 

Proof  Lorward:  Assume  /  is  a  best  response  to  {qk}.  Then  /  minimizes  Y^k= i  P r(k)qkA/I  f 
Therefore  it  also  minimizes  Pr {k)qkM f  —  Pr (k)qkc  because  Pr {k)qkc  is  constant 

with  respect  to  /.  Backward  direction  is  similar.  □ 

Even  though  the  adversary  knows  his  type  (i.e.,  the  correct  value  of  k)  he  cannot  use  LP  ?? 
to  find  his  equilibrium  strategy  because  the  sender  does  not  know  the  exact  value  of  k.  The 
adversary’s  equilibrium  strategy  can  instead  be  computed  by  the  dual  to  LP  11. 
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LP  12  Equilibrium  Sender  Strategy 

in  a  Bayesian  game  (uncertain  M  and  c) 

Input:  G,  {M1},  {c},  k,  L,  Pr(i) 
Output:  /,  {R1},  {A'} 

Minimize  w  f  +  'y  Pr(i) 

/.{#}.{  \ 
1=1 

kR  +  y  a;  ) 

^  aeA  / 

(5.69) 

subject  to: 

K  >  row a[My  -  <  -  A* 

Vo  g  A, Vi  e  [1..L] 

(5.70) 

^  ^  fvu  ^  ^  fuv 

(v,u)€E  (u,v)eE 

\/veV\  {t} 

(5.71) 

fuv  >  0 

V(m,  v)  e  E 

(5.72) 

A?a>0 

Vo  e  A,  Vi  e  [1..L] 

(5.73) 

5.6.2  Uncertain  Payoffs 

Another  way  in  which  the  sender  may  be  uncertain  of  the  adversary  is  by  not  knowing  the 
payoffs  and  costs.  Suppose  instead  that  he  has  a  probability  distribution  over  L  possible 
payoff  matrices  and  attack  costs  (types  of  adversaries).  For  i  e  [1  ..L],  let  Pr(i)  be  the 
probability  that  the  adversary  is  of  type  i  with  a  harm  matrix  M%  and  cost  of  attacks  c\  The 
sender’s  optimal  equilibrium  strategy  is  computed  by  LP  12.  As  before,  the  adversary’s 
strategy  can  be  computed  by  taking  the  dual  of  this  LP. 

If  we  assign  Pr(i)  =  1  for  every  i  e  [1  ,.L\  we  get  a  linear  program  which  solves  an¬ 
other  interesting  variant  of  our  problem.  Consider  a  game  with  one  sender  and  multiple 
adversaries.  The  adversaries  choose  their  strategies  independently  of  each  other  (i.e.,  no 
colluding).  The  adversaries  have  different  harm  matrices  and  costs  for  attacking  nodes  and 
the  total  harm  to  the  sender  is  the  sum  of  the  harm  resulting  from  each  adversary’s  attack. 
The  payoff  to  each  adversary  depends  only  on  his  own  strategy  and  the  sender’s  strategy; 
it  does  not  depend  on  the  strategies  of  any  of  the  other  adversaries.  For  now,  let’s  assume 
that  every  adversary  can  attack  k  nodes.  By  assigning  Pr(i)  =  1  for  every  i  e  [1  ,.L\  we 
get  that  LP  12  computes  the  equilibrium  strategy  for  the  sender  in  the  multiple  adversaries 
game  too!  As  for  the  attacker’s  equilibrium  strategies,  we  get  an  interesting  observation: 
since  the  strategies  can  be  computed  by  the  dual  of  LP  12,  they  are  in  fact  correlated. 
Even  though  the  adversaries  choose  their  strategies  independently  of  each  other,  due  to  the 
strategic  consideration  they  behave  as  if  they  coordinate  their  moves. 
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5.6.3  Experiments 


In  this  section  we  experimentally  evaluate  our  algorithms.  We  empirically  evaluated  our 
approach  through  simulation  of  two  multiagent  systems,  a  multirobot  team  operating  in  an 
urban  environment  and  a  sensor  network  transmitting  through  a  multihop,  ad  hoc  network. 
Each  data  point  is  the  average  over  20  independent  randomly  generated  instances. 

Source  nodes  in  the  sensor  network  were  sensor  nodes  that  were  generating  data  (for  ex¬ 
ample,  readings  from  a  target)  that  had  to  be  transmitted  back  for  fusion  at  one  of  several 
base  stations  (the  sink  nodes).  Other  nodes  in  the  network  were  used  as  relays  between  the 
sensors  and  the  base  stations.  Nodes  were  distributed  uniformly  at  random  in  a  10  x  10 
region  of  the  plane.  Edges  were  added  between  nodes  within  a  Euclidean  distance  of  1  of 
each  other,  yielding  a  unit  disk  graph,  a  commonly  used  abstraction  of  wireless  network 
topologies.  This  resulted  in  networks  that  on  average  had  many  paths  between  any  two 
nodes,  but  were  still  not  fully  connected  (in  which  case  the  problem  is  trivial). 

The  graph  in  the  robot  team  represented  an  urban  road  network,  with  source  nodes  repre¬ 
senting  the  starting  locations  of  the  robots.  We  conducted  experiments  on  both  synthetic, 
square  grid  graphs.  We  also  used  topologies  extracted  from  GIS  data  2  of  the  road  network 
of  Afghanistan  as  recorded  by  the  Afghanistan  Information  Management  Service.  The 
graph  is  shown  in  Figure  5.8. 

Source  nodes  were  selected  uniformly  at  random  from  among  the  nodes,  and  sink  nodes 
were  selected  uniformly  at  random  from  among  the  nodes  excluding  the  source  nodes.  For 
the  presented  experiments  |T|  =  10  and  k  —  10. 

We  used  a  commercial  LP  solver,  Gurobi  4.6.1,  running  on  a  64-bit  Windows  7  computers 
with  a  2.8  GHz,  second-generation  Intel  Core  processor  and  8  GB  of  RAM. 

The  first  set  of  experiments  examined  the  impact  of  varying  the  number  and  size  of  groups. 
One  might  expect  that  having  the  source  nodes  divided  into  more  groups  would  increase 
the  running  time  as  there  would  be  more  possible  combinations  of  sinks  chosen  for  each 
group.  To  investigate  this,  we  varied  the  number  of  groups  from  one  to  four  while  inde¬ 
pendently  varying  the  size  of  each  group  from  one  to  ten.  This  resulted  in  instances  where 
the  total  number  of  source  nodes  varied  from  one  to  forty. 

Figures  5.9  and  5.10  show  the  average  running  times  plotted  for  the  total  number  of  source 
nodes,  using  log-log  axes.  Note  that  the  points  for  all  groups  are  roughly  collinear.  This 
shows  that  it  is  the  total  number  of  source  nodes,  not  the  specific  distribution  of  the  source 
nodes  into  groups,  that  affects  running  time.  Also  note  that  the  points  in  both  plots  show 
a  linear  trend,  which  because  of  the  log-log  plot  indicates  that  the  running  time  increases 

2http://www.mapcruzin.com/free-afghanistan-roads-arcgis-maps-shapefiles.htm 
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Figure  5.9:  Running  time  of  LP  10  vs.  number  of  source  nodes  as  the  number  of  number 
of  groups  and  number  of  source  nodes  per  group  is  varied  for  the  grid  network 
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Figure  5.10:  Running  time  of  LP  10  vs.  number  of  source  nodes  as  the  number  of  number 
of  groups  and  number  of  source  nodes  per  group  is  varied  for  the  disk  network. 


polynomially  with  the  number  of  source  nodes,  as  expected  for  a  linear  programming- 
based  approach.  For  the  grid  network,  the  best  fit  line  has  a  slope  of  2.0247  (with  R 2  = 
0.9816)  while  for  the  disk  network  the  best  fit  line  has  a  slope  of  2.0945  (with  R 2  = 
0.9855),  indicating  that  the  effect  of  the  number  of  source  nodes  on  running  time  is  close 
to  quadratic. 

The  next  set  of  experiments  looked  at  the  impact  of  the  size  of  the  graph  on  running  time 
by  varying  the  number  of  nodes  for  each  type  of  graph.  For  the  grid  graph  we  varied  the 
size  of  the  graph  from  100  to  10,000  nodes.  For  the  disk  graph  we  varied  the  number  of 
nodes  that  we  placed  from  100  to  10,000  nodes  as  well;  however,  due  to  the  stochastic 
nature  of  network  formation,  not  all  nodes  were  connected  in  the  every  resulting  instance 
which  could  cause  problems  if  source  in  the  same  group  could  not  reach  the  same  sink. 
For  each  instance  we  extracted  the  largest  connected  component  and  used  this  as  the  graph 
of  the  environment.  The  average  number  of  nodes  in  these  graphs  varied  from  81  to  9608. 
For  the  graph  of  the  road  network  of  Afghanistan,  we  isolated  rectangular  geographical  re¬ 
gions  of  varying  sizes  around  Kabul  and  extract  the  subgraph  induced  by  the  nodes  within 
the  rectangular  regions.  Again  this  could  result  in  some  nodes  being  disconnected  (for 
example,  because  their  connections  in  the  road  network  extended  outside  of  the  specific 
rectangular  region  we  were  considering)  and  so  we  took  the  largest  connected  component 
for  each  region  size  as  the  graph.  The  number  of  nodes  in  the  Afghanistan  maps  varied 
from  123  to  12,459. 
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Grid  B  Disk  A  Afghanistan 


Figure  5.11:  Running  time  of  LP  10  vs.  number  of  nodes. 


The  running  times  vs.  the  number  of  nodes  for  the  three  topologies  are  shown  in  Fig¬ 
ure  5.11  using  a  log-log  plot.  The  grid  network  has  the  highest  average  running  times 
overall,  followed  by  the  disk  network,  and  then  the  Afghanistan  network.  The  points  show 
a  linear  trend,  with  slopes  ranging  from  1.58  for  grid  network  to  1.42  for  the  Afghanistan 
network,  indicating  polynomial  running  time  that  is  sub-quadratic.  This  shows  that  LP  10 
is  scalable  for  large  numbers  of  nodes. 

We  also  plot  the  running  times  against  the  number  of  edges.  This  is  important  as  the 
number  of  paths  through  the  network  depends  not  just  on  the  number  of  nodes  but  also 
on  the  number  of  edges.  The  results  are  shown  in  Figure  5.12,  again  using  logarithmic 
axes.  We  again  see  linear  trends,  this  time  with  slopes  ranging  from  1.55  for  the  grid 
network  to  1.41  for  the  Afghanistan  graph.  Surprisingly,  the  running  times  on  the  disk  and 
Afghanistan  graphs  are  roughly  collinear,  while  the  running  times  on  the  grid  network  are 
significantly  higher.  This  is  interesting  because  the  disk  network  has  more  edges  overall, 
from  90  to  53,893,  while  the  grid  network  has  between  360  and  39,600  edges  and  the 
Afghanistan  network  between  308  and  33,846  edges. 

Our  final  set  of  experiments  compares  our  approach  with  two  heuristics.  Minimum  Harm 
(MH)  minimizes  the  possible  harm  that  can  be  caused  by  the  adversary  but  does  not  con¬ 
sider  the  sender’s  environmental  costs.  Shortest  Paths  (SP)  minimizes  the  environmental 
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Grid  B  Disk  A  Afghanistan 


Figure  5.12:  Running  time  of  LP  10  vs.  number  of  edges. 


cost  but  ignores  any  possible  attack  by  an  adversary.  We  vary  the  relative  weight  of  the 
harm  and  environmental  costs  in  the  sender’s  payoff  according  to  the  formula 

^sender ( f  >  <l)  =  ~qM f  ~  jwf,  (5.74) 

where  7  >  0  is  a  constant  sender  cost  factor  that  we  vary  from  2~6  to  22  in  powers  of  2. 

Figures  5.13  -5.15  shows  the  total  costs  to  the  sender  (the  sum  of  the  harm  and  envi¬ 
ronmental  costs)  when  the  adversary  plays  a  worst  case  equilibrium  strategy  on  different 
graph  topologies.  This  adversary  strategy  does  not  depend  on  the  sender’s  choice  of  strat¬ 
egy.  Note  that  the  sender  plays  off-equilibrium  when  he  plays  MH  and  SP,  and  hence  the 
strategy  profiles  are  not  in  equilibrium.  Indeed,  the  adversary’s  strategy  may  not  even  be 
a  best  response  to  the  sender’s  strategy,  as  it  was  chosen  assuming  that  the  sender  would 
play  an  equilibrium  strategy. 

From  the  figures  it  is  clear  that  the  EQ  and  SP  strategies  perform  similarly,  while  the  MH 
strategy  does  increasingly  poorly  as  the  sender  cost  factor  increases.  This  is  to  be  expected, 
as  the  MH  strategy  does  not  factor  in  the  environmental  costs  at  all,  instead  choosing  to 
divide  the  flow  over  as  many  paths  as  possible  to  reduce  exposure  to  attack,  even  if  those 
paths  are  very  long.  As  7  is  increased,  the  environmental  costs  come  to  dominate  the 
sender’s  costs  and  so  MH  does  poorly.  From  these  results  it  seems  that  SP  is  superior  to 


101 


Sender  Performance  Against  Worst  Case 
Equilibrium  Adversary:  Grid 

-■-EQ  -4-MH  -A-SP 


Sender  Cost  Factor  (logarithmic) 


Figure  5.13:  Total  costs  to  the  sender  when  playing  an  equilibrium  (EQ),  minimum  harm 
(MH),  or  shortest  paths  (SP)  strategy  against  an  adversary  playing  a  worst  case  equilibrium 
strategy  on  a  grid. 


MH. 

However,  there  is  also  the  issue  of  security.  When  playing  MH  or  SP  the  sender  is  playing 
off-equilibrium.  If  the  adversary  knows  that  the  sender  is  playing  off-equilibrium,  there 
is  no  reason  for  him  to  continue  to  play  an  equilibrium  strategy  of  his  own.  Instead,  he 
will  play  a  strategy  that  will  maximize  his  own  payoff  against  the  sender’s  off-equilibrium 
play.  We  consider  the  adversary  playing  a  worst  case  best  response,  that  is,  a  best  response 
that  has  the  worst  payoff  for  the  sender.  This  reflects  the  case  where  the  adversary  can 
either  observe  the  sender’s  actual  strategy  or  knows  which  type  of  strategy  the  sender  is 
playing  (MH  or  SP).  It  also  establishes  the  security  of  the  game  of  the  sender,  that  is,  the 
worst  case  outcome  against  a  rational  adversary. 

The  total  costs  to  the  sender  when  playing  against  an  adversary  playing  worst  case  best 
response  is  shown  in  Figures  5.16-5.18.  In  Figure  5.16  we  clearly  see  that  the  SP  strategy 
does  very  poorly  compared  to  EQ  and  even  MH  except  for  high  values  of  7.  The  reason 
for  this  is  that  SP  is  very  vulnerable  to  attack:  it  chooses  a  single  path  through  the  network, 
which  means  the  adversary  can  choose  a  pure  strategy  that  maximizes  the  harm. 

This  is  effect  still  present  for  the  disk  and  Afghanistan  graphs,  but  it  is  more  difficult  to  see 
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Sender  Performance  Against  Worst  Case 
Equilibrium  Adversary:  Disk 


-■-EQ  -#-MH  -A-SP 


Figure  5.14:  Total  costs  to  the  sender  when  playing  an  equilibrium  (EQ),  minimum  harm 
(MH),  or  shortest  paths  (SP)  strategy  against  an  adversary  playing  a  worst  case  equilibrium 
strategy  on  a  disk  network. 


due  to  the  environmental  costs  dominating  the  total  costs,  especially  for  the  MH  strategy. 
This  makes  the  relative  contribution  from  SP’s  greater  vulnerability  to  attack  harder  to 
see.  Careful  examination  of  Figure  5.17  shows  that  there  is  a  significant  gap  between  the 
curves  for  EQ  and  SP,  but  this  cannot  be  visually  seen  in  Figure  5.18. 

A  better  way  to  visualize  this  vulnerability  is  to  consider  the  relative  security  gap.  The 
security  gap  is  the  difference  in  payoffs  to  the  sender  when  the  adversary  plays  a  worst 
case  equilibrium  strategy  and  when  the  adversary  plays  a  worst  case  best  response.  The 
relative  security  gap  then  normalizes  this  to  the  sender’s  payoff  against  the  equilibrium  ad¬ 
versary.  Thus,  the  relative  security  gap  reflects  the  sender’s  vulnerability  as  the  percentage 
difference  in  payoff  between  an  equilibrium  adversary  and  a  best  response  adversary. 

The  relative  security  gap  is  plotted  in  Figures  ??  -  ??  for  the  three  topologies;  a  more 
negative  value  indicates  worse  performance.  We  can  see  that  the  SP  strategy  performs 
poorly  in  all  three  topologies  for  small  values  of  7.  As  7  increases,  the  gap  narrows  as  the 
total  costs,  and  hence  the  denominator,  increase.  Still,  it  is  very  clear  from  these  graphs 
that  the  increase  in  total  costs  for  MH  is  due  entirely  to  the  rising  environmental  costs  as 
7  increases;  MH  has  a  small  security  gap  and  hence  is  not  very  vulnerable  to  attack,  as  is 
expected. 
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Sender  Performance  Against  Worst  Case 
Equilibrium  Adversary:  Afganistan 


-■-EQ  -♦-MH  -±-SP 


Sender  Cost  Factor  (logarithmic) 


Figure  5.15:  Total  costs  to  the  sender  when  playing  an  equilibrium  (EQ),  minimum  harm 
(MH),  or  shortest  paths  (SP)  strategy  against  an  adversary  playing  a  worst  case  equilibrium 
strategy  on  the  Afghanistan  road  network. 


From  this  set  of  experiments  we  can  conclude  that  the  equilibrium  strategies  found  by 
LP  10  offers  the  best  of  both  heuristics:  it  provides  the  security  of  MH  and  the  good 
performance  of  SP,  trading  off  the  risks  of  harm  and  environmental  costs  in  an  intelligent 
way.  This  is  the  goal  of  our  work  on  flow  allocation  in  adversarial  environments,  and  our 
approach  meets  that  goal. 
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Sender  Performance  Against  Worst  Case  Best 
Response  Adversary:  Grid 


-B-EQ  -*-MH  -A-SP 


Sender  Cost  Factor  (logarithmic) 


Figure  5.16:  Total  costs  to  the  sender  when  playing  an  equilibrium  (EQ),  minimum  harm 
(MH),  or  shortest  paths  (SP)  strategy  against  an  adversary  playing  a  worst  case  best  re¬ 
sponse  strategy  on  a  grid. 

5.7  Stackelberg  Games 


In  this  section  we  consider  the  Stackelberg  game  in  which  the  sender  plays  first,  commit¬ 
ting  to  a  strategy.  We  show  how  two  commonly  used  solution  concepts,  the  strong  and 
weak  Stackelberg  equilibria,  are  inappropriate  for  sequential  network  security  games,  and 
provide  a  polynomial  time  algorithm  for  finding  a  more  nuanced  equilibrium. 


5.7.1  Model 

In  the  previous  section  we  described  the  simultaneous  game  where  the  sender  and  adver¬ 
sary  act  without  observing  each  other’s  actions.  However,  in  many  settings  this  is  not 
the  case.  For  example,  convoys  in  support  of  persistent  military  or  humanitarian  relief 
missions  will  operate  over  extended  periods  of  time  and  the  adversary  can  observe  routes 
taken  over  time  to  build  up  an  estimate  of  the  sender’s  mixed  strategy  before  choosing 
which  attacks  to  launch.  These  types  of  settings  are  commonly  modeled  as  Stackelberg 
games,  a  type  of  sequential  game  in  which  one  player  (the  “leader”)  moves  first,  commit¬ 
ting  to  a  mixed  strategy.  The  second  player  (the  “follower”)  can  then  observe  that  mixed 
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Sender  Performance  Against  Worst  Case  Best 
Response  Adversary:  Disk 

-■-EQ  -4-MH  -±-SP 


Figure  5.17:  Total  costs  to  the  sender  when  playing  an  equilibrium  (EQ),  minimum  harm 
(MH),  or  shortest  paths  (SP)  strategy  against  an  adversary  playing  a  worst  case  best  re¬ 
sponse  strategy  on  a  disk  network. 


strategy  and  choose  an  appropriate  response.  It  is  known  that  in  Stackelberg  games  the 
leader  can  sometimes  improve  his  equilibrium  payoff  (and  cannot  decrease  it,  under  mild 
assumptions)  compared  to  his  equilibrium  payoff  in  the  simultaneous  move  game  [59]. 

In  a  two-player  Stackelberg  game  the  follower’s  strategy  is  a  function  that  maps  mixed 
strategies  of  the  leader  to  mixed  strategies  of  the  follower.  In  the  network  flow  security 
game  with  attack  costs,  the  adversary’s  strategies  are  functions  g  :  T  — *  A  that  map 
each  flow  to  an  adversary  mixed  strategy.  Let  Q  denote  the  set  of  all  such  functions. 
A  Stackelberg  equilibrium  is  a  refinement  of  subgame  perfect  Nash  equilibrium  where 
( f*,g *)  are  a  Stackelberg  equilibrium  if  they  are  mutual  best  responses,  that  is 

g\t)Mt  =  max  g*(f)Mf 

g*{f*)Mf*  -  g*(r)c  =  ma xg(r)Mf*  -  s(/*)c. 

see 

both  hold,  and  g*(f)  is  a  best  response  to  /  for  all  /  e  T  (the  follower  always  plays 
optimally,  even  off  the  equilibrium  path).  Computing  a  best  response  function  g  for  the 
adversary  is  straightforward:  given  /,  greedily  choose  up  to  k  attacks  that  have  maximum 


106 


Sender  Performance  Against  Worst  Case  Best 
Response  Adversary:  Afganistan 


HB-EQ  -*-MH  -±-SP 


Figure  5.18:  Total  costs  to  the  sender  when  playing  an  equilibrium  (EQ),  minimum  harm 
(MH),  or  shortest  paths  (SP)  strategy  against  an  adversary  playing  a  worst  case  best  re¬ 
sponse  strategy  on  the  Afghanistan  road  network. 


Relative  Security  Gap:  Grid 


-B-EQ  -4-MH  -±-SP 
Sender  Cost  Factor 

0.01  0.1  1  10 


Figure  5.19:  Relative  security  gap  when  the  sender  plays  an  equilibrium  (EQ),  minimum 
harm  (MH),  or  shortest  paths  (SP)  strategy  on  a  grid  network. 
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Relative  Security  Gap:  Disk 


-■-EQ  -4-MH  -A-SP 
Sender  Cost  Factor 

0.01  0.1  1  10 


Figure  5.20:  Relative  security  gap  when  the  sender  plays  an  equilibrium  (EQ),  minimum 
harm  (MH),  or  shortest  paths  (SP)  strategy  on  a  disk  network. 


Relative  Security  Gap:  Afganistan 


-■-EQ  -♦-MH  -A-SP 
Sender  Cost  Factor 


0.01  0.1  1  10 


Figure  5.21:  Relative  security  gap  when  the  sender  plays  an  equilibrium  (EQ),  minimum 
harm  (MH),  or  shortest  paths  (SP)  strategy  on  the  Afghanistan  road  network. 
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payoff  to  the  adversary,  excluding  any  that  would  contribute  negative  payoff  because  the 
attack  cost  is  too  high.  Note  that  there  will  be  multiple  best  response  functions  if  there 
is  some  /  for  which  the  set  of  k  attacks  yielding  highest  adversary  payoff  is  not  unique, 
and  that  these  best  response  functions  may  yield  different  payoffs  to  the  sender  because  of 
heterogeneous  attack  costs.  Thus  there  may  be  multiple  Stackelberg  equilibria  that  have 
the  same  sender  strategy  but  different  sender  payoffs. 

Traditionally  two  kinds  of  Stackelberg  equilibrium  are  distinguished:  strong  Stackelberg 
equilibrium  (SSE),  where  the  follower’s  best  response  function  always  maps  to  a  strategy 
that  maximizes  the  leader’s  payoff;  and  weak  Stackelberg  equilibrium  (WSE),  where  the 
follower’s  best  response  function  always  maps  to  a  strategy  that  minimizes  the  leader’s 
payoff  [34].  The  pessimistic  WSE  is  the  more  natural  solution  concept  for  security  appli¬ 
cations,  which  tend  to  focus  on  worst  case  behavior.  Despite  this,  SSE,  which  assumes 
that  the  malicious  adversary  breaks  ties  in  the  leader’s  favor,  has  been  considered  more 
often  in  the  literature  for  two  technical  reasons:  (1)  a  SSE  is  guaranteed  to  exist  in  every 
Stackelberg  game,  while  a  WSE  may  not;  and  (2)  it  is  often  claimed  that  the  leader  can 
induce  the  adversary  to  play  the  desired  best-case  strategy  by  deviating  by  an  arbitrarily 
small  amount  from  the  equilibrium  in  order  to  break  the  adversary’s  indifference  [59].  We 
will  show  that  both  of  these  arguments  are  inappropriate  for  the  network  security  game, 
but  first  illustrate  several  important  concepts  by  example. 

Recall  the  example  in  Figure  5.1  with  ci  =  102.  The  adversary  is  indifferent  when  j\  = 
103/105,  prefers  the  top  path  when  it  is  /i  >  103/105,  and  prefers  the  bottom  path  when 
fi  <  103/105.  Thus  all  best  response  functions  g\  :  [0, 1]  — *  [0, 1]  mapping  fi  to  the 
probability  of  attacking  the  top  path  must  satisfy  g\(fi)  =  0  when  f\  <  103/105  and 
9i{fi)  —  1  when  f\  >  103/105,  and  any  value  g\(fi)  €  [0,1]  is  acceptable  for  f\  = 
103/105. 

In  the  unique  simultaneous  Nash  equilibrium,  p1  =  3/105,  so  that  the  sender  was  in¬ 
different  between  the  top  and  bottom  paths  but  sent  f\  =  103/105  flow  on  the  top  path 
and  f-2  =  2/105  flow  on  the  bottom  path,  suffering  harm  on  both  paths.  It  follows  that 
fi  =  103/105  is  a  best  response  to  the  adversary’s  best  response  function  g\!E  with 
(7^(103/105)  =  3/105.  Thus  the  simultaneous  Nash  equilibrium  naturally  gives  rise 
to  a  Stackelberg  equilibrium  strategy,  the  same  payoff  to  the  sender  as  in  the  Nash  equilib¬ 
rium,  —306/105.  In  the  SSE  the  adversary  attacks  the  bottom  path  (gf'S£'(103/105)  =  0), 
resulting  in  a  much  higher  payoff  to  the  sender,  —6/105.  It  is  easy  to  see  that  there  are 
no  other  Stackelberg  equilibria  for  this  game.  For  example,  there  is  no  WSE  because  if 
the  adversary  played  the  worst-case  best  response  with  5f/’ors*(103/105)  =  1,  then  the  the 
sender  would  have  incentive  to  deviate  by  decreasing  f\ . 

The  sender’s  strategy  is  the  same  in  both  of  these  equilibria  which  means  that  his  payoff 
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Figure  5.22:  A  network  topology  in  which  the  sender  cannot  induce  a  strong  Stackelberg 
equilibrium. 


ultimately  depends  on  the  choice  of  the  indifferent  adversary.  However,  note  that  the 
sender  can  deviate  slightly  from  his  equilibrium  strategy  by  playing  f\  =  103/105  —  £  for 
some  small  e  >  0,  in  order  to  incentivize  the  adversary  to  attack  the  bottom  path.  By  doing 
this  the  sender  will  receive  a  payoff  of  6/105  +  3e  instead  of  the  6/105  that  he  would  earn 
in  the  SSE,  but  as  e  is  made  arbitrarily  small  his  strategy  converges  to  the  SSE  strategy. 

It  is  not  always  possible  to  induce  the  SSE  by  deviating  from  an  equilibrium  strategy. 
Consider  the  network  in  Figure  5.22,  and  assume  that  A  contains  two  attacks,  one  that 
affects  e\  and  one  that  affects  e 2,  with  harm  matrix 


and  costs  c\  =  3  and  c2  =  1.  The  sender  has  no  choice  as  his  only  pure  strategy  is  to  send 
the  full  flow  on  the  single  path  from  s  to  t.  At  the  same  time,  the  adversary  is  indifferent 
to  the  choice  of  attack  as  they  both  yield  him  a  payoff  of  2  and  so  might  choose  either  of 
them. 


5.7.2  Inducing  Locally  Optimal  Equilibria 

Given  that  the  WSE  may  not  exist  and  the  SSE  might  not  be  attainable,  we  solve  the  prob¬ 
lem  of  how  the  sender  can  deviate  from  a  Stackelberg  equilibrium  strategy  /  to  induce  a 
Stackelberg  equilibrium  (/,  g)  that  yields  him  maximum  payoff.  We  call  this  equilibrium  a 
locally  optimal  induceable  Stackelberg  equilibrium  (loptlSE).  It  is  locally  optimal  because 
the  value  of  the  Stackelberg  equilibrium  that  is  induced  depends  on  the  starting  equilibrium 
strategy  /.  The  starting  strategy  that  we  use  is  one  that  arises  naturally  from  simultaneous 
game  Nash  equilibrium  strategy  as  solved  for  using  LP  9,  which  can  be  shown  to  always 
be  a  Stackelberg  equilibrium: 

Lemma  6.  A  strategy  profile  ( f,p )  is  a  Nash  equilibrium  for  the  network  flow  security 
game  with  attack  costs  if  and  only  if  if  ■  g)  is  a  Stackelberg  equilibrium  for  the  Stackelberg 
network  flow  security  game  with  attack  costs  for  a  best  response  function  g  with  g(f)  =  p. 
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Algorithm  5  Computing  deviation  to  find  optimal  inducible  Stackelberg  equilibrium 
1:  Find  Nash  equilibrium  flow  /  using  LP9 

2:  Set  A'  to  be  the  set  of  minimum  adversary  payoff  candidate  attacks. 

3:  Set  k!  <k  to  be  the  number  of  candidate  attacks  that  must  be  chosen  from  A! . 

4:  if  \A'\  <  k'  then 
5:  Return. 

6:  Add  dummy  source  so  to  G.  Set  /'  0.  Set  f£  to  be  the  empty  flow. 

7:  while  \I\  <  k'  do 

8:  Set  F  <-  0. 

9:  for  all  a  e  A'  do 

10:  Solve  (fa,  H,  H')  LP13 

11:  if  H  -  H'  >  0  then 

12:  Fl-FU  {fa} 

13:  Set  Y  {a\fa  e  F  with  minimum  row a[M]fa}. 

14:  Set  Z  {a|3a'  G  Y  s.t.  row a[M]fa  =  rowa/[M]/a/} 

15:  Set  A'  <-  A' \Z  and 

16:  Set  f^f  +  Eaey/° 

17:  Set  f^f  +  ef* 

18:  for  all  s  E  S  do 

19:  Normalize  outgoing  flow. 


Proof.  Suppose  (f,p)  is  a  Nash  equilibrium  and  construct  g  as  a  best  response  function 
with  g(f)  =  p.  By  definition  of  Nash  equilibrium  /  and  g(f)  are  are  mutual  best  responses 
because  and  so  (/,  g)  is  a  Stackelberg  equilibrium.  Suppose  (/,  g)  are  a  Stackelberg  equi¬ 
librium.  Then  by  definition  of  Stackelberg  equilibrium  /  and  g(f)  =  p  are  mutual  best 
responses  and  hence  (/,  p)  is  a  Nash  equilibrium.  □ 


Algorithm  5  computes  a  deviation  from  equilibrium  strategy  that  the  sender  can  use  to  in¬ 
duce  a  loptlSE.  We  will  first  sketch  the  high  level  approach  before  delving  into  the  details. 
The  algorithm  starts  from  a  Nash  equilibrium  flow  /  that  will  also  be  a  Stackelberg  equi¬ 
librium  strategy  according  to  Lemma  6.  It  then  computes  the  set  A'  of  candidate  attacks 
for  the  adversary.  These  are  the  attacks  that  might  be  chosen  as  part  of  a  best  response 
to  /.  The  sender  will  try  to  incentivize  the  adversary  to  choose  certain  of  these  candidate 
attacks  by  adding  small  amounts  of  flow  to  certain  paths.  Intuitively  this  approach  exploits 
what  we  observed  in  the  example:  parallel  paths  allow  the  sender  freedom  to  deviate  and 
bias  the  adversary;  a  sequential  topology  does  not  permit  that  flexibility.  However,  the 
process  is  not  as  obvious  when  dealing  with  general  attacks,  each  of  which  may  affect 
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an  arbitrary  set  of  links  with  heterogeneous  penalties.  Instead  of  choosing  a  simple  path, 
the  sender  tries  to  find  a  flow  for  each  candidate  attack  that  will  cause  the  adversary  to 
prefer  to  play  that  attack.  If  the  attacks  are  “in  parallel”  this  will  be  possible,  if  the  attacks 
are  “in  sequence”  it  will  not  be.  When  presented  with  multiple  options,  the  sender  will 
choose  the  one  that  causes  the  least  harm  (i.e.,  increases  his  payoff  the  most).  The  process 
repeats  until  the  sender  has  guided  all  of  the  adversary’s  attacks,  or  he  has  guided  all  the 
attacks  that  are  possible.  The  deviation  flows  (which  may  be  made  arbitrarily  small)  are 
then  superimposed  on  the  original  flow  to  generate  the  desired  deviation. 

Computing  the  candidate  attacks  is  straightforward.  Given  /,  we  compute  the  payoff  that 
the  adversary  would  receive  for  each  attack  a  €  A  as  pa  =  row a[M]f .  If  there  are  k  or 
fewer  attacks  with  pa  >  0,  then  those  are  all  candidate  attacks.  If  there  are  more  than  k 
such  attacks,  we  compute  A'  as  the  set  of  attacks  with  the  highest  k  values  of  pa  (with 
repetition).  For  example,  if  the  set  of  pa  is  {10,10,8,7,7,7,0,-2}  and  k  =  4,  then 
A'  =  {a  G  A\ra  >  7},  giving  us  6  candidate  attacks.  The  adversary’s  best  response 
will  always  choose  the  attacks  with  pa  strictly  greater  than  the  minimum,  so  we  need  only 
consider  A'  to  be  those  with  minimum  pa  values.  In  the  previous  example,  that  means 
that  the  best  response  always  plays  the  two  attacks  with  pa  =  10  and  the  one  attack  with 
pa  =  8,  so  we  are  left  to  choose  k'  =  k  —  3  =  1  candidate  attacks  from  among  the  three 
remaining  with  pa  =  7. 

A  dummy  source  node  s0  is  added  to  G  and  connected  to  each  source  in  s  G  S',  allowing 
deviant  flows  from  any  source.  /,  the  set  of  induced  attacks,  is  initialized  as  empty.  The 
overall  deviant  flow  fe  is  initially  empty. 

The  algorithm  then  iterates  up  to  k!  times  in  the  loop  starting  at  line  7.  On  each  iteration 
it  attempts  to  greedily  induce  the  adversary  to  choose  attacks  that  maximally  increase  the 
sender’s  payoff.  F  is  the  set  of  best  deviant  flows,  computed  for  each  candidate  attack 
in  the  loop  starting  at  line  9.  The  best  deviant  flow  for  a  candidate  attack  a  is  computed 
by  LP13.  This  LP  finds  a  flow  that  causes  as  great  as  possible  an  increase  in  harm  for 
attack  a  compared  to  any  other  candidate  attack.  For  a  deviation  flow  /“,  the  adversary’s 
best  response  is  an  attack  with  maximum  increase  in  harm  (attack  cost  does  not  matter  as 
the  adversary  is  already  indifferent  between  candidate  attacks  due  to  the  Nash  equilibrium 
sender  strategy),  so  if  the  objective  value  is  non-negative,  the  adversary  can  be  induced  to 
play  a  (and  possibly  other  candidate  attacks  as  well).  If  the  objective  value  is  negative,  the 
adversary  cannot  yet  be  induced  to  play  a  in  preference  to  other  candidate  attacks. 

Of  the  attacks  that  can  be  induced  on  this  iteration,  the  sender  chooses  those  that  cause 
minimum  increase  in  harm.  These  may  not  be  unique,  so  Y  is  the  set  of  all  such  candidate 
attacks  with  minimum  increase  in  harm  that  the  adversary  can  be  induced  to  attack  on  this 
iteration.  The  deviation  flows  that  are  used  to  induce  these  attacks  may  also  induce  other 
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LP  13  Stackleberg  deviating  flow  for  a. 

Input:  G,  M,  A',  a 

Output:  /“,  H,  H' 


Maximize  H  —  H' 

(5.75) 

subject  to: 

H'  >  row a[M]fa 

(5.76) 

H  —  row  a'[M]fa' 

\fa  e  A'  \  {a} 

(5.77) 

^  ]  fvu  =  ^  ^  fuv 

Vn  G  V  \  {s0,t} 

(5.78) 

(v,u)gE  (u,v)eE 

fs0U  —  1 

(5.79) 

( so,u)eE 

fur,  >  0 

V(u,v)  G  E 

(5.80) 

Aa  >  0 

Va  G  A 

(5.81) 

attacks  (which  have  the  same  increase  in  harm),  so  the  set  Z  contains  all  the  attacks  that 
will  be  induced  in  the  current  iteration.  These  are  removed  from  the  candidate  attacks 
and  added  to  the  induced  attacks  in  line  15,  and  the  deviation  flows  for  this  iteration  are 
superimposed  on  the  total  deviation  flow  f£  before  starting  a  new  iteration. 

The  loop  terminates  when  the  requisite  number  of  attacks  have  been  induced.  The  case 
when  no  more  candidate  attacks  can  be  induced  is  handled  by  there  being  a  single  flow  in 
F  which  is  trivially  deviated  from  itself.  For  performance  considerations  this  possibility 
can  be  checked  separately.  It  is  also  not  possible  for  A'  to  become  empty  prior  to  the 
termination  of  the  loop.  Recall  also  that  at  the  beginning  of  iteration,  \A'\  >  k!  (lines  4  - 
5)  and  on  every  iteration  the  same  number  of  attacks  are  added  to  /  as  are  removed  from 
A! .  Thus,  |/|  >  k'  no  later  than  the  iteration  when  A'  =  0. 

In  line  17  the  deviation  flow  is  scaled  and  superimposed  on  the  equilibrium  flow,  and  in 
line  19  the  amount  of  flow  (which  increased  due  to  the  addition  of  the  deviation  flow)  is 
normalized  at  each  source  node  so  that  the  total  amount  of  flow  is  maintained  with  the 
addition  of  the  deviation. 

Theorem  5.  Algorithm  5  runs  in  time  polynomial  in  the  size  of  G  and  A. 

Proof.  Each  line  in  the  algorithm  can  clearly  be  executed  in  polynomial  time.  The  for 
loops  in  lines  9-12  and  lines  18-19  iterate  at  most  0(|v4|)  and  0(n)  time,  respectively. 
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In  each  iteration  of  the  main  loop  from  lines  7  -  16  at  least  1  attack  is  added  to  |/|  and 
therefore  the  loop  cannot  iterate  more  than  |/|  =  0(|^4|)  times.  □ 
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Chapter  6 
Related  Work 

6.1  Flow  Allocation 


The  issue  of  transmitting  data  to  a  central  base  station  has  received  considerable  attention 
in  ad  hoc  and  especially  sensor  networks  [29,  28,  39,  1].  The  primary  concern  for  many 
algorithms  operating  in  these  kinds  of  networks  is  power  usage,  and  to  a  lesser  extent, 
scalability  and  simplicity.  This  has  led  to  many  algorithms  that  give  rise  to  tree  or  tree-like 
topologies,  such  as  through  hierarchical  or  cluster-based  routing  [28,  39],  data  aggrega¬ 
tion  [1,  19],  or  topology  control  [49].  By  focusing  on  tree  topologies,  we  can  address 
partial  centralization  problems  without  committing  to  a  specific  network  algorithm. 

A  number  of  well-known  network  design  problems  are  also  related  to  the  flow  allocation 
problems  we  consider  in  this  paper.  Given  an  undirected  graph,  a  root  node,  a  set  of 
source  nodes  and  their  demands,  and  a  uniform  edge  capacity,  the  capacitated  minimum 
Steiner  tree  problem  (CMStT)  is  to  find  the  minimum  cost  Steiner  tree  in  which  all  source 
nodes  can  route  their  flows  to  the  root  without  violating  the  edge  capacity  constraints.  A 
{''<(.) st  +  2) -approximation  was  given  in  [33],  where  7  is  the  Steiner  ratio  and  pST  is  the 
best  achievable  approximation  algorithm  for  the  Steiner  tree  problem.  There  are  several 
key  differences  from  the  flow  allocation  problem.  First,  CMStT  seeks  to  minimize  edge 
costs  for  routing  all  flows,  while  our  objective  is  to  maximize  the  number  of  source  nodes 
whose  full  flows  can  routed  to  the  sink.  Second,  CMstT  assumes  uniform  edge  capacities, 
while  in  general  we  do  not. 

In  the  unsplittable  flow  problem  (UFP)  you  are  given  a  graph  and  source-sink  pairs  along 
with  a  given  demand,  and  the  problem  is  to  find  a  single  path  from  each  source  node  to  its 
sink  along  which  its  flow  can  be  routed.  There  are  three  variants,  all  NP-hard:  maximizing 
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the  amount  of  flow  transmitted,  partitioning  the  sources  into  the  minimum  rounds  such 
that  all  sources  within  a  round  can  transmit  their  flows  simultaneously,  and  minimizing  the 
maximum  congestion  along  an  edge  in  the  network.  Several  approximation  algorithms  for 
these  problems  have  been  devised  [36,  11].  However,  none  of  these  consider  the  problem 
where  groups  of  source  nodes  must  transmit  to  a  common  sink.  Our  proof  of  Theorem  2 
follows  a  proof  from  Guruswami,  et  al.  [26]  of  the  hardness  of  approximating  the  edge 
disjoint  paths  problem  (EDP)  of  finding  a  path  from  each  source  to  its  paired  destination, 
without  any  of  the  paths  sharing  an  edge  in  common. 


6.2  Network  Augmentation 

Network  augmentation  combines  three  fundamental  problems:  supplemental  node  deploy¬ 
ment,  the  assignment  of  groups  to  sinks,  and  flow  allocation  on  the  edges.  There  has  been 
considerable  research  related  to  each  of  these  problems  individually,  but  never  for  all  three 
of  them  simultaneously.  The  network  augmentation  problem  is  very  similar  to  facility 
location  problems  that  have  been  extensively  studied  in  the  operations  research  commu¬ 
nity  [40,  67,  5,  45].  Given  a  number  of  facilities  and  a  set  of  sites  with  quantities  of  goods 
demanded  at  each  site,  the  facility  location  problem  is  to  place  the  facilities  in  order  to 
satisfy  the  demand  of  the  sites  for  minimum  transportation  costs  of  goods  from  facilities 
to  sites.  Variants  to  the  facility  location  problem  include  capacitated  problems  [14,  45] 
in  which  there  are  capacity  constraints  on  facilities  or  on  transportation  links,  as  well  as 
multiple  commodity  problems  [54]  where  different  facilities  can  provide  different  types  of 
goods.  One  major  difference  between  facility  location  problems  and  the  network  augmen¬ 
tation  problem  is  that  they  assume  a  direct  link  between  facility  and  site,  while  communi¬ 
cation  from  group  member  to  sink  can  be  through  a  multihop  network.  Another  is  that  the 
flows  from  groups  of  source  nodes  must  be  transmitted  to  the  same  sink. 

One  area  of  networking  research  focuses  on  how  to  provide  access  between  two  initially 
disconnected  networks  through  the  addition  of  additional  hubs,  routers,  and  bridges.  The 
problems  addressed  in  that  literature  are  similar  to  the  network  augmentation  problem. 
One  approach  is  to  partition  the  network  into  multiple  local  access  networks  (LANs)  and 
a  backbone  network  [2,  56].  Each  LAN  has  one  node  that  is  designated  the  access  point 
and  is  also  part  of  the  backbone  network.  Traffic  between  nodes  in  different  LANs  must 
first  be  routed  to  the  access  point  for  the  originating  LAN,  conveyed  across  the  backbone 
network,  and  then  routed  across  the  destination  LAN.  Links  are  considered  costly  and  ca¬ 
pacitated.  The  local  access  network  design  problem  is  to  design  the  local  access  networks 
by  purchasing  LAN  edges  between  nodes  so  that  the  total  cost  is  minimized  and  a  known 
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amount  of  traffic  can  be  routed  to  the  access  points,  which  has  similarities  to  both  network 
augmentation  and  flow  allocation.  The  access  network  design  problem  is  NP-hard,  but 
it  is  known  that  there  exist  optimal  solutions  in  which  the  LANs  take  the  form  of  trees 
with  the  access  points  as  roots  [2].  Linear  programming  formulations  have  been  used  to 
approximately  solve  the  access  network  design  problem  in  [2]  and  more  recently  in  [56]. 
Unlike  the  network  augmentation  problem  considered  here,  however,  all  inter-LAN  traffic 
must  be  conveyed  through  the  backbone  network.  In  addition,  the  communication  pattern 
is  single-source-single-destination,  instead  of  the  centralizing  groups  we  consider  here. 

The  formation  of  two-tiered  communication  networks  has  also  been  a  focus  in  mobile  ad 
hoc  networking  [4,  23,  12].  In  these  networks,  there  is  no  pre-existing  “backbone  net¬ 
work”  and  the  problem  is  to  dynamically  create  such  a  network  from  the  underlying  ad 
hoc  network.  This  is  accomplished  by  partitioning  the  nodes  into  clusters,  and  selecting 
a  clusterhead  node  in  each  cluster  to  act  as  the  access  node  for  the  backbone  network. 
As  result,  all  intra-cluster  traffic  is  conducted  in  multi-hop  through  the  ad  hoc  network, 
but  inter-cluster  traffic  must  pass  through  the  clusterhead,  which  then  relays  it  to  other 
clusterheads.  The  primary  focus  of  research  has  been  in  developing  techniques  of  clus¬ 
ter  formation  and  clusterhead  selection,  such  as  highest  ID  [4],  highest  degree  [23],  node 
weight  [6],  and  weighted  clustering  [12].  These  techniques  primarily  focus  on  metrics 
such  as  cluster  stability  and  power  conservation,  and  ignore  communication  costs,  capac¬ 
ities,  and  specific  communication  requirements.  In  addition,  they  assume  that  the  nodes 
(including  the  clusterheads)  move  exogenously,  while  we  actively  position  of  the  supple¬ 
mental  nodes  specifically  to  meet  the  demands  of  the  network. 

Virtual  private  network  (VPN)  provisioning  [25]  is  one  of  the  few  areas  that  consider  a 
group  communication  pattern.  In  VPN  provisioning,  groups  of  nodes  within  a  network 
wish  to  form  a  subnetwork  by  reserving  bandwidth  from  the  underlying  network.  Given 
bounds  on  the  communication  demands  of  the  nodes  that  wish  to  form  the  VPN,  the  VPN 
provisioning  problem  is  to  reserve  bandwidth  so  that  any  traffic  pattern  respecting  the 
given  bounds  can  be  feasibly  routed.  Polynomial-time  optimal  and  approximation  algo¬ 
rithms  were  found  for  some  problems,  but  the  capacitated  version  of  the  problem  is  NP- 
hard  [25].  The  “group  communication”  in  VPNs  differ  from  that  in  this  thesis  because 
VPN  member  communicate  with  each  other,  while  centralizing  group  members  transmit 
data  to  the  centralization  point.  Also,  while  VPN  provisioning  allocates  bandwidth  for  a 
group-oriented  communication  pattern,  it  does  not  address  the  issue  of  supplementing  the 
network  through  additional  nodes. 
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6.3  Security  in  Adversarial  Environments 


Choosing  paths  through  hostile  environments  have  been  studied  in  operations  research  [66, 
30],  robotics  [27,  7],  and  multiagent  systems  [62,  32].  Many  of  these  have  also  taken 
the  perspective  of  the  player  who  selects  nodes  or  edges  in  the  network  to  impair  the 
other  player  who  chooses  paths  through  the  network.  The  study  of  network  interdiction 
[66,  30]  looks  at  problems  where  an  interdictor  chooses  edges  or  nodes  to  damage  or 
destroy  destroy  (“interdict”)  in  order  to  impair  the  ability  of  an  enemy  moving  through  the 
network,  for  example  for  by  forcing  it  to  take  longer  paths  [30].  An  early  study  of  single 
source,  single  sink  zero-sum  games  where  the  interdictor  interdicts  a  single  edge  found 
that  the  equilibrium  strategy  is  to  only  interdict  edges  in  the  minimum  cut  [66].  Similar 
results  were  found  in  network  routing  settings  [10],  and  more  recently  in  games  where 
multiple  edges  can  be  interdicted[62,  32].  In  evader-pursuer  games  [27,  7],  both  players 
move  through  the  network.  In  path  disruption  games  [3]  multiple  cooperative  agents  work 
together  to  interdict  an  adversary,  in  contrast  to  our  setting  where  both  sides  are  assumed 
to  be  monolithic  players. 

In  most  of  these  related  problems,  the  payoff  depends  on  the  probability  that  at  least  one 
attack  occurs  on  a  pathway;  multiple  attacks  on  the  same  pathway  either  are  not  possible 
or  incur  no  additional  penalty.  This  models  situations  like  placing  checkpoints  to  intercept 
the  sender;  once  caught,  the  sender  cannot  be  caught  again.  In  contrast,  in  our  problem 
the  same  pathway  may  be  subject  to  multiple  attacks  or  a  single  attack  may  affect  multiple 
edges  on  the  same  pathway,  resulting  in  additional  harm.  This  is  useful  for  settings  where 
the  sender  continues  after  an  attack,  as  when  convoys  fight  their  way  through  ambushes 
or  robots  clear  obstacles.  Games  with  similar  payoffs  have  been  solved  in  the  context  of 
Markov  Decision  Processes  using  oracle  algorithms  [43].  However,  these  approaches  have 
assumed  only  zero-sum  games. 

A  class  of  security  games  between  two  players,  an  attacker  and  a  defender,  have  recently 
been  proposed  and  studied  in  a  variety  of  contexts  [52,  64,  34,  69,  62,  32,  63].  The  attacker 
chooses  targets  to  attack  from  a  known  set  of  targets.  The  defender  attempts  to  foil  attacks 
by  assigning  defensive  resources  from  a  known  set  of  defensive  resources  to  a  known  set  of 
schedules,  where  there  is  a  known  subset  of  schedules  that  each  resource  can  be  assigned 
to.  Each  schedule  protects  or  covers  a  subset  of  the  targets.  For  each  target  that  is  attacked, 
the  attacker  and  defender  receive  payoffs  depending  on  whether  the  target  is  covered  or 
not,  and  when  multiple  targets  are  attacked  the  payoff  is  the  sum  of  the  payoffs  for  each 
target.  A  key  characteristic  of  these  security  games  is  that  the  defender  receives  a  higher 
payoff  when  a  target  is  covered  than  when  it  is  uncovered  and  the  attacker  receives  a  higher 
payoff  when  a  target  is  uncovered  than  when  it  is  covered.  Crucially  to  comparison  with 
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this  thesis,  the  payoffs  for  a  target  is  the  same  if  is  attacked  one  or  more  times,  or  if  it  is 
covered  by  one  or  more  resources. 

Yin  et  al.  [69]  studied  the  equilibrium  properties  of  these  types  of  security  games,  examin¬ 
ing  the  relationships  between  minimax  strategies,  Nash  equilibria,  and  strong  Stackelberg 
equilibria  for  Stackelberg  games  where  the  defender  is  the  leader  and  the  attacker  is  the  fol¬ 
lower.  They  proved  that  when  the  attacker  can  only  attack  a  single  target,  the  defender’s 
minimax  and  Nash  equilibrium  strategies  are  identical,  and  that  all  Nash  equilibria  are 
interchangeable.  Furthermore,  given  the  additional  assumption  that  any  subset  of  a  de¬ 
fender’s  schedule  is  itself  a  schedule,  they  showed  that  the  defender’s  SSE  strategies  are 
also  Nash  equilibrium  strategies. 

There  are  many  similarities  between  these  security  games  and  the  games  considered  in 
this  thesis.  However,  careful  examination  reveals  that  the  games  in  this  thesis  do  not  sat¬ 
isfy  the  assumptions  of  the  security  games  as  defined  by  Yin  et  al.  and  others.  The  key 
difference  is  in  the  way  payoffs  are  computed  in  the  two  models.  In  Yin  et  al.,  payoffs 
are  computed  based  on  the  binary  conditions  of  whether  the  target  is  attacked  or  not  at¬ 
tacked,  and  covered  or  uncovered.  In  this  thesis,  the  central  component  of  the  payoff,  the 
harm,  depends  on  the  specific  attack  and  path  chosen  for  the  flow,  with  harm  summed  over 
multiple  attacks  and  paths. 

We  can  demonstrate  this  fundamental  difference  with  a  simple  example  of  a  zero-sum 
game.  Because  the  security  games  of  Yin  et  al.  are  played  over  finite  strategy  spaces,  a 
natural  comparison  is  with  the  zero-sum  path  game  (ZS-PG)  of  this  thesis,  with  network 
flows  then  understood  to  compactly  represent  mixed  strategies  over  the  combinations  of 
paths.  Consider  the  graph  with  a  single  source  node,  a  single  sink  node,  and  three  paths 
from  the  source  to  the  sink,  as  shown  in  Figure  6.1.  The  sender  has  three  pure  strategies: 
7Ti,  the  path  through  iq;  7t2,  the  path  through  v2,  and  7 r3,  the  path  through  the  v3. 

Suppose  that  the  adversary  has  three  attacks,  A  =  {ai,a2,a3},  and  that  he  can  play  a 
single  attack  at  a  time  (i.e.,  k  —  1)  so  that  he  also  has  three  pure  strategies.  Each  attack 
affects  two  edges,  causing  harm  according  to  the  following  harm  matrix: 


H 


2  1  0  0  0  0 
0  2  1  0  0  0 
1  0  2  0  0  0 


(6.1) 


Note  that  the  effect  on  the  harm  is  unequal  between  the  two  edges  affected  by  each  attack. 
For  example,  when  the  adversary  plays  attack  a±,  the  sender  suffers  2  units  of  harm  if  he 
plays  7Ti  (which  includes  ei)  and  only  1  unit  of  harm  if  he  plays  n2  (which  includes  e2). 

We  can  represent  the  payoffs  explicitly  using  the  normal  form,  where  the  sender  is  the  row 
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Figure  6.1:  Graph  for  counter-example  of  NFG  as  a  security  game. 


player  and  the  adversary  is  the  column  player.  This  gives  us  the  following  payoff  table: 


-2,2 

o 

o' 

-1,1 

-1,1 

-2,2 

o 

o' 

o 

o' 

-1,1 

-2,2 

These  payoffs  do  not  satisfy  the  structure  of  a  security  game  as  defined  by  Yin  et  al. 
because  it  cannot  be  defined  in  terms  of  covered  and  uncovered  targets  that  are  either 
attacked  or  not.  For  example,  when  the  adversary  plays  a\,  the  sender  can  receive  one  of 
three  possible  payoffs:  -2,  -1,  or  0.  That  cannot  be  represented  as  an  attacked  target  being 
either  covered  or  uncovered  (which  would  lead  to  only  two  possible  payoffs).  Nor  does 
any  pure  strategy  dominate  any  others,  allowing  us  to  effectively  reduce  the  strategy  spaces 
to  ones  where  we  can  represent  this  game  as  a  security  game.  This  example  shows  that  the 
games  studied  in  this  thesis,  even  the  simplest  zero-sum,  single-source,  single-sink  games, 
are  not  a  subset  of  the  security  games  previously  addressed  in  the  literature,  and  thus  the 
theoretical  results  that  have  been  established  in  that  literature  is  not  directly  applicable  to 
our  games. 

Instead  we  can  look  to  the  concept  of  strategically  zero-sum  games  defined  by  Moulin 
and  Vial  [44].  These  are  a  generalization  of  zero-sum  games  that  share  many  properties 
of  zero-sum  games,  such  as  equilibria  strategies  being  minimax  strategies  and  equilibria 
being  interchangeable.  Lemma  2  of  this  thesis  establishes  that  the  non- zero-sum  network 
flow  game  is  strategically  equivalent  (as  defined  by  Moulin  and  Vial)  to  the  zero-sum 
game  with  payoffs  defined  by  U (/,  q)  in  Equation  (5.1),  thus  meeting  the  definition  of  a 
strategically  zero-sum  game. 
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Stackelberg  games  [50,  62]  have  recently  been  used  to  model  security  games  where  pat¬ 
terns  of  behavior  may  be  observed  and  learned  by  the  adversary,  as  opposed  to  more  tra¬ 
ditional  simultaneous  games[10,  42].  Stackelberg  games  generally  allow  the  leader  to 
find  equilibrium  strategies  with  higher  payoff  than  in  a  simultaneous  game,  but  only  in 
non-zero  sum  games  [69,  59].  Computing  the  optimal  strategies  to  commit  to  is  solvable 
in  polynomial  time  in  the  normal  form  game  [13],  but  this  is  not  practical  in  our  games 
which  have  exponential-sized  strategy  spaces.  The  traditional  solution  concept  considered 
in  all  of  these  is  the  strong  Stackelberg  equilibrium,  which  is  questionable  for  the  worst- 
case  reasoning  common  in  security  settings  and  is  not  appropriate  for  the  network  security 
games  we  consider. 
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Chapter  7 

Conclusion  and  Future  Work 


This  thesis  addressed  the  problem  of  flow  allocation  to  support  multiagent  task  execution. 
Performing  tasks  requires  inputs  to  be  brought  to  the  task  execution  location:  performing 
a  physical  task  requires  agents  and  resources  to  move  through  the  physical  environment 
to  the  location  of  the  task,  while  performing  a  computational  task  requires  input  data  to 
be  transmitted  through  the  communication  network  to  a  computational  platform  where  the 
computation  can  occur.  This  thesis  modeled  the  movement  of  these  inputs  as  network 
flows,  formulated  and  analyzed  problems  in  allocating  flows  in  several  kinds  of  environ¬ 
ments,  and  presented  algorithms  for  solving  these  problems.  This  chapter  summarizes  the 
major  results  of  this  thesis  and  discusses  some  lines  of  future  work. 


7.1  Summary 

This  thesis  addressed  the  problem  of  flow  allocation  in  three  environmental  contexts. 
Chapter  2  formulated  the  flow  allocation  problem  for  known,  capacitated  environments. 
This  approach  was  extended  in  Chapter  3  to  capacitated  environments  that  could  be  aug¬ 
mented  by  the  agents  to  add  supplemental  nodes  and  edges.  Chapters  4  and  5  addressed 
flow  allocation  in  costly  environments  where  costs  were  affected  by  an  adversary. 


7.1.1  Capacitated  Environments 

Chapter  2  formulated  the  flow  allocation  problem  using  a  directed,  capacitated  graph  for 
the  environment,  groups  of  source  nodes  for  agent  subteams,  and  sink  nodes  for  task  ex- 
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ecution  locations.  The  core  flow  allocation  problem  is  to  assign  sinks  to  groups  and  find 
flows  from  the  source  nodes  to  the  sink  assigned  to  their  groups. 

This  thesis  makes  three  main  contributions: 

1.  Maximizing  the  number  of  groups  transmitting  their  full  amount  of  flow  to  their  as¬ 
signed  sink  is  formulated  as  the  Maximum  Satisfied  Group  (MaxSG)  problem.  This 
corresponds  to  maximizing  the  number  of  tasks  that  can  be  executed.  When  flows 
can  be  divided  on  multiple  paths,  as  arises  with  data  streams  in  communication  net¬ 
works  or  some  kinds  of  resources  in  physical  networks,  MaxSG  on  general  graphs 
is  strongly  NP-hard  and  so  there  are  no  pseudo-polynomial  time  algorithms  known 
for  solving  it  optimally. 

2.  A  pseudo-polynomial  time  algorithm  for  solving  MaxSG  with  divisible  flows  for 
graphs  with  tree  topologies,  as  arise  in  some  kinds  of  communication  networks, 

3.  Proof  that  when  flows  cannot  be  divided  among  multiple  paths,  for  example  with 
embodied  agents  that  must  take  a  single  path  from  the  source  to  the  sink,  flow  al¬ 
location  is  not  only  NP-hard  to  solve  optimally,  but  NP-hard  to  approximate  better 
than  satisfying  the  requirements  of  a  single  group. 

7.1.2  Network  Augmentation 

Chapter  3  considered  flow  allocation  in  capacitated  graphs  where  the  agents  could  aug¬ 
ment  the  network  by  adding  supplemental  nodes  and  edges.  This  models  communication 
networks  that  can  be  enhanced  through  the  addition  of  additional  dedicated  relay  nodes, 
or  physical  environments  where  new  movements  can  be  enabled  through  the  removal  of 
obstructions.  The  potential  network  represented  the  locations  where  supplemental  nodes 
could  be  deployed  and  the  edges  that  would  result  if  they  were.  Two  main  problems 
were  addressed:  maximizing  the  number  of  satisfied  groups  with  a  fixed  number  of  sup¬ 
plemental  nodes  (MaxSG-NA)  and  minimizing  the  number  of  supplemental  nodes  while 
satisfying  all  groups  (MinDep).  Solving  either  of  these  required  finding  deployments  of 
supplemental  nodes  to  potential  locations  (network  augmentation),  as  well  as  finding  sink 
assignments  and  flows  from  the  source  nodes  to  the  assigned  sinks  (flow  allocation). 

This  thesis  makes  three  main  contributions: 

1.  Augmenting  a  network  to  provide  connectivity  to  source  node  groups  is  NP-hard. 

2.  Mixed  integer  linear  programs  for  solving  the  NP-hard  problems  MaxSG-NA  and 
MinDep  optimally. 

3.  Heuristics  for  MinDep  and  MaxSG-NA  that  iterate  through  the  groups  and  extended 
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the  supplemental  node  deployments  and  sink  assignments  as  each  group  was  con¬ 
sidered.  These  heuristics  were  empirically  shown  to  dramatically  decrease  running 
time  at  the  expense  of  a  modest  decrease  in  solution  quality  when  compared  to  the 
optimal  algorithms. 


7.1.3  Adversarial  Environments 

Chapters  4  and  5  considered  flow  allocation  in  costly  graphs  where  the  costs  were  partially 
chosen  by  an  adversary.  In  this  setting  divisible  flows  can  directly  represent  the  movement 
of  inputs  that  can  be  divided  among  multiple  paths  (like  communication  traffic),  or  may 
represent  probability  distributions  over  the  movements  of  indivisible  inputs  (like  robots). 
We  modeled  this  setting  as  a  two-player  game  between  a  sender  (representing  the  agent 
team)  who  chose  sink  assignments  and  flows,  and  an  adversary  who  choose  multiple  at¬ 
tacks  that  imposed  flow  costs  on  edges.  The  main  problem  addressed  was  to  compute 
equilibrium  strategies  for  the  sender. 

This  thesis  makes  three  main  contributions: 

1.  A  polynomially-sized  linear  program  to  find  equilibrium  strategies  in  the  zero-sum 
setting  where  the  payoffs  for  the  sender  and  the  adversary  were  directly  opposed. 

2.  A  proof  that  a  similar  approach  can  be  used  in  a  class  of  non-zero-sum  games  where 
the  players  unilaterally  incur  costs  based  on  their  chosen  strategies,  and  a  linear 
program  for  the  generalized  problem  with  multiple  sinks  and  multiple  groups  with 
these  kinds  of  payoffs. 

3.  An  example  that  existing  solution  concepts  are  inadequate  for  settings  where  the 
sender  must  commit  to  a  strategy  that  is  then  observed  by  the  adversary,  and  a  new 
equilibrium  refinement,  the  optimal  inducible  Stackelberg  equilibrium,  to  address 
this  shortcoming,  and  an  algorithm  for  computing  a  sender  strategy  for  it. 


7.2  Future  Work 

In  this  section  we  describe  several  areas  of  future  work  for  flow  allocation. 
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7.2.1  Generalized  Task  Structures 


This  thesis  considered  relatively  simple  task  structures:  each  task  required  inputs  from 
a  known  set  of  agents  and  sets  of  agents  performed  a  single  task.  In  many  multiagent 
applications  the  task  structures  are  considerably  more  complex.  Computational  tasks  often 
exhibit  hierarchical  structure,  with  the  outputs  of  computations  at  lower  levels  being  used 
as  inputs  to  computations  at  a  higher  level.  For  example,  hierarchical  team  plans  impose 
a  hierarchy  on  plan  monitoring  tasks.  The  plan  monitoring  task  for  a  team  of  agents 
executing  a  team  plan  does  not  need  to  know  the  exact  status  of  each  agent  in  the  team. 
Instead,  it  can  use  status  information  from  the  plan  monitors  for  each  subteam  of  agents 
performing  a  subplan.  In  sensor  networks,  the  output  of  a  computational  task  like  object 
recognition  can  then  be  used  as  input  to  another  computational  task,  like  threat  assessment. 

With  physical  tasks,  teams  of  agents  may  have  to  perform  multiple  tasks.  Even  if  physical 
tasks  require  the  full  attention  of  the  agents  performing  them,  precluding  the  agents  from 
carrying  out  multiple  tasks  simultaneously.  However,  the  agents  may  still  perform  multiple 
tasks  over  time  as  they  complete  a  series  of  tasks,  one  at  a  time.  For  example,  a  team  of 
firefighters  may  work  to  put  out  a  single  burning  building,  but  once  that  fire  is  extinguished, 
they  are  able  to  move  to  the  another  burning  building. 

A  direction  for  future  research  is  to  extend  the  flow  allocation  approach  taken  in  this  thesis 
to  these  kinds  of  cases.  One  way  could  be  to  allow  sink  nodes  that  receive  flow  to  act  as 
source  nodes  for  new  flows.  For  computational  tasks,  this  new  flow  would  represent  the 
output  of  the  computation,  and  need  not  be  equal  to  the  amount  of  flow  (i.e.,  input  data) 
that  was  consumed  by  the  sink.  For  physical  tasks,  a  new  flow  would  need  to  be  created 
for  each  source  node  that  transmitted  to  the  sink,  to  represent  the  agent  that  moved  to  the 
task  location. 


7.2.2  Dynamic  Teams,  Tasks,  and  Environments 

Dynamism  is  a  major  challenge  in  many  multiagent  applications.  All  aspects  of  a  multi¬ 
agent  system  may  change  over  time:  the  agents  (or  their  organization  into  subteams),  the 
tasks,  and  the  environment.  In  the  context  of  flow  allocation,  this  means  that  groups  of 
source  nodes,  the  set  of  sink  nodes,  and  the  graph  itself  may  change  over  time. 

A  crude  way  to  deal  with  dynamism  is  to  re-run  the  algorithms  when  things  change,  but 
this  is  often  unsatisfactory  for  many  reasons.  It  can  be  computationally  impractical  to 
repeatedly  re-run  the  algorithms,  especially  when  the  changes  may  not  result  in  a  change 
in  the  solution,  as  when  the  communication  network  changes  slightly  due  to  the  movement 
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of  the  agents.  When  the  solution  does  change,  small  changes  in  the  inputs  may  result  in 
very  large  changes  to  the  outputs,  which  impose  hidden  costs,  as  when  a  deployment  of 
supplemental  nodes  changes  dramatically.  Finally,  such  a  scheme  is  inherently  reactive 
and  myopic,  which  can  decrease  performance  over  time. 

It  may  be  possible  to  address  these  shortcomings  by  adapting  the  algorithms  in  this  thesis. 
Further  research  may  determine  what  kinds  of  changes  require  a  solution  (or  part  of  a  solu¬ 
tion)  to  be  recomputed.  It  could  also  be  possible  to  mitigate  the  costs  of  of  changes  in  the 
solution,  perhaps  by  explicitly  representing  those  costs  in  the  optimization.  A  third  pos¬ 
sibility  would  be  to  make  the  algorithms  more  proactive,  for  example  by  using  stochastic 
programming  [15,  9]  to  reason  about  the  effects  of  dynamism  provided  probabilistic  infor¬ 
mation  is  available. 

7.2.3  Partially  Distributed  Algorithms 

The  algorithms  in  this  thesis  are  centralized  approaches  that  rely  on  full  knowledge  of  the 
environment.  A  natural  next  step  is  to  develop  distributed,  parallel  algorithms  to  compute 
flow  allocations  that  do  not  require  full  knowledge  of  the  environment  at  any  single  lo¬ 
cation.  This  could  result  in  more  scalable  algorithms  that  do  not  require  the  full  graph 
of  the  environment  to  be  transmitted  to,  stored  at,  or  computed  over  at  any  single  loca¬ 
tion.  It  could  improve  running  time  through  parallel  computation  and  reduced  input  sizes. 
Although  some  degree  of  distribution  is  desirable,  it  is  not  necessary  for  approaches  to 
be  fully  distributed.  In  many  distributed  multiagent  coordination  algorithms  (for  example 
distributed  constraint  optimization)  it  is  beneficial  to  use  some  amount  of  centralization  in 
order  to  reduce  communication  and  redundant  computation. 

Many  multiagent  applications  feature  strong  locality.  Agents  are  much  more  likely  to  have 
relatively  good  knowledge  of  their  local  physical  and  communication  environments  than 
they  are  to  know  the  full  environment.  Agents  are  also  often  located  near  each  other  in 
the  environment,  and  nearby  task  execution  locations  are  more  preferable  to  more  distant 
ones,  all  other  things  being  equal.  Because  of  this,  it  may  be  possible  to  compute  high 
quality  local  solutions  based  primarily  on  local  knowledge,  with  a  need  to  communicate 
global  knowledge  (in  the  form  of  agents,  tasks,  or  topological  information)  only  in  a  small 
number  of  cases.  The  key  will  be  to  efficiently  exploit  the  locality  while  also  being  able  to 
handle  the  rare  but  often  (and  often  crucial)  exceptions. 
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